How do I troubleshoot traps without a line number?

[mx270a]

I have an app running on a PK70 quad serial that is occasionally throwing traps several hours after boot up. Today's trap was after 35 hours of up time. The trap information I'm getting is:

Code: Select all

-------------------Trap information-----------------------------
Exception Frame/A7 =0210FCB8
Trap Vector        =Access Error (2)
Format             =04
Status register SR =2000
Fault Status       =08
Faulted PC         =0206C13E

-------------------Register information-------------------------
A0=00000000 A1=200009C4 A2=0203F764 A3=0206E2D2
A4=021351C0 A5=0203F764 A6=0210FCC8 A7=0210FCB8
D0=00000000 D1=00002000 D2=0208B3AC D3=0208B3AC
D4=00000001 D5=0000000B D6=02114FAA D7=0203F764
SR=2000 PC=0206C13E
-------------------RTOS information-----------------------------
The OSTCBCur current task control block = 20000A9C
This looks like a valid TCB
The current running task is: HTTP#2D
-------------------Task information-----------------------------
Task    | State    |Wait| Call Stack
Idle#3F|Ready     |    |0200C04A,0200BF20,0
Main#32|Ready     |    |0200DCE2,02001F68,0200BF20,0
TCPD#28|Semaphore |0002|0200D912,0201C0B6,0200BF20,0
IP#27|Fifo      |0006|0200CDE8,02010B84,0200BF20,0
Enet#26|Fifo      |0024|0200CDE8,0202DEA8,0200BF20,0
FTPD#30|Semaphore |0000|0200D912,0201F144,02017338,0200BF20,0
TELNET#37|Semaphore |0000|0200D912,0201BC6C,020023A8,0200BF20,0
HTTP#2D|Running   |    |0206C13E,0
QUADSERIAL#2A|Semaphore |0001|0200D912,0201F144,02005144,0200BF20,0

-------------------End of Trap Diagnostics----------------------

I used WinAddr2Line to see what the location 0206C13E shows, and it says this:

Code: Select all

std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)
??:0

I see the HTTP task is the one that was running at the time. I have a web browser displaying a page from this device with an AJAX section that pulls data every 1 second, so I expect the HTTP task to be doing stuff.

So how do I figure out what is causing this random problem?

Thanks,
Lance

[dciliske]

I'd check every single address register to see if you can trace the call. Looking at the registers and the trap vector, it appears to be a NULL pointer dereference (D0 register). Also, it appears you're not getting a line number due to it being a StdLib issue (looks like that's what's being called). I could be wrong, but I think the StdLib is stripped of debugging symbols, as it is not recompiled as a general rule. This would account for the missing line number.

...though this would appear to mean that it's trapping in a StdLib function, i.e. potentially a bug in StdLib. Great -.-

That said, it's also possible something somewhere is corrupting a stack, causing the issue.

-Dan

P.S. sorry for rambling, it's morning and I appear to need my coffee.

[tod]

Might be helpful to post some code.

My stab in the pitch dark is you're running out of stack or heap. Since it takes 35 hours I would guess heap. Is it possible you have a memory leak? Are you newing up a string in the code on the NB that handles the Ajax request? By default on NB, the new is nothrow, so you have to check that the value returned is not null. Without code I'm just rounding up the usual suspects.

[mx270a]

It wasn't a consistent 35 hours. I had seen crashes in 4-8 hours as well. I'm not "new"ing any variables in the HTTP section, and the minimum free stack space was staying the same when I checked it in TaskScan. The randomness is certainly confusing me.

As this appears to be related to STD, I went through the code that AJAX function calls to find all instances of STD to see if I could remove them. I had 12 places with code like this:

Code: Select all

siprintf(output, "%s", std::string(str_SomeVar).c_str());

but I discovered the std::string part is not necessary to convert a string to a c_str for use in a siprintf. Ended up changing them all to:

Code: Select all

siprintf(output, "%s", str_SomeVar.c_str());

Hopefully this completely eliminates the issue, although if it does, I'm still going to be curious what was causing this trap. As of right now, the system has been up for 10 hours, so I need to wait a couple more days to see if it does anything odd.

Thanks,
Lance

[rnixon]

I will second Tod's memory theory. Whenever I have had a trap without a line number it was because my code had a memory problem: bad pointer, task stack corruption, running out of task stack space, writing beyond the end of a buffer, etc.

Autovars get put on the task stack, check to see your not making large buffers as autovars. If so, make them static so they go in global space, or make them global.

Could also be running out of heap space. If your using dynamic memory calls, are you checking all return values?

[mx270a]

Could also be running out of heap space. If your using dynamic memory calls, are you checking all return values?

I must admit that sentence is greek to me. I'll have to do some google-fu to learn about heap space and dynamic memory calls.

[tod]

I don't think you fixed anything. your additional std::string(str_SomeVar) was just making a copy of the string. It is good to get rid of it since you didn't need it but given that strings are reference counted it would have a minimal impact and shouldn't cause a trap.

How possible is it that "output" could be overrun? That is, is it guaranteed to always be one character larger than str_someVar? I realize you didn't show us all the code so maybe you have checks for this. How did you create output?

Unless there is something proprietary about this code why don't you just post it? That's way easier than making us guess what you did. If it's long, use https://gist.github.com/ and post the link.

Tod

P.S. Dynamic memory calls are just things like new (in C++) and malloc (in C). They allocate memory from the heap (aka the free store) as opposed to the global memory space or the stack. The thing about dynamic memory calls is you're also responsible for releasing the memory: delete or delete[] in C++ or free() in C.

[mx270a]

Here is the code from my web.cpp file as it is currently. Since the active task at the time of the crash was the HTTP task, I think the issue is/was in this file. The web page with the AJAX call was running the getAJAXSystemStatus function with a 1 second refresh.

Code: Select all

#include <startnet.h>
#include <stdio.h>
#include <iostream>
#include <string>
#include <math.h>
#include <autoupdate.h>
using namespace std;

//Define External Variables
extern const char* AppName;
extern unsigned long g_UptimeSeconds;
extern bool g_USUnits;

extern int g_DirectionTTL;
extern bool g_DirectionForward;
extern string g_IOCardVersion;
extern bool g_MaxOfflineDistanceOverrideActive;

extern int g_ReceiverTTL[];
extern double g_ReceiverDistance[2][2];
extern double g_ReceiverHeading[];
extern string g_ReceiverFields[2][12];
extern double g_Distances[2][2];

extern double g_WayPointHeading[];
extern double g_WayPointDistance[];
extern bool g_WayPointStraight[];
extern int g_WayPointCount;
extern int g_ActiveWayPointSegment;
extern double g_ActiveWayPointHeading;

char output[2000];
char fnum[20];

// Functions called from a web page must be declared extern C
extern "C" {
   void DisplayFirmwareVersion(int sock, PCSTR url);
   void DisplayIOCardVersion(int sock, PCSTR url);
   void getAJAXSystemStatus(int sock, PCSTR url);
   void getAJAXPathList(int sock, PCSTR url);
   void RebootSystem();
   void ReadUserSettings();
   void WriteUserSettings();
   void DisplayUnitsDropDown(int sock, PCSTR url);
}

// Structure for storing and retieving from user Flash
#define VERIFY_VALUE (0x48666050) //A random value picked from the phone book....
struct MyOwnDataStore {
      DWORD verify_key;
      bool USUnits;
};
void ReadUserSettings() { //Read the stored data
   MyOwnDataStore *pData = (MyOwnDataStore*) GetUserParameters();
   //Verify it has the right key value. In a "Real" application, a checksum would be more robust.
   if (pData->verify_key == VERIFY_VALUE) { //Data is ok
	   if (pData->USUnits) {
		   g_USUnits = true;
		   iprintf("Loading System Settings: Using US units (feet).\r\n");
	   } else {
		   g_USUnits = false;
		   iprintf("Loading System Settings: Using metric units (meters).\r\n");
	   }
   } else {
	   iprintf("System settings could not be read from NVRAM.\r\n");
   }
}
void WriteUserSettings() {
	MyOwnDataStore mds;
	mds.USUnits = g_USUnits;
	mds.verify_key = VERIFY_VALUE;
	if (SaveUserParameters( &mds, sizeof(mds)) != 0) {
		iprintf("Parameters saved in User Parameter area.\r\n");
	} else {
		iprintf("Programming failed \r\n");
	}
}



int MyDoPost(int sock, char *url, char *pData, char *rxBuffer) {
	iprintf("----- Processing Post -----\r\n");
	iprintf("Post URL: %s\r\n", url);
	iprintf("Post Data: %s\r\n", pData);

	if (httpstricmp(url+1, "REBOOTSYSTEM")) {
		//ExtractPostData("textForm1", pData, textForm1, MAX_BUF_LEN);
		//iprintf("textForm1 set to: \"%s\"\r\n", textForm1 );
		RebootSystem();
		// Tell the web browser to issue a GET request for the next page
		RedirectResponse(sock, "advanced.htm");

	} else if (httpstricmp(url+1, "SETUNITS")) {
		char newunits[20];
		if (ExtractPostData("units", pData, newunits, 20) != -1) {
			if (!strcmp(newunits, "metric")) {
				g_USUnits = false;
				iprintf("Setting units to Metric.\r\n");
			} else if (!strcmp(newunits, "us")) {
				g_USUnits = true;
				iprintf("Setting units to Feet.\r\n");
			} else {
				iprintf("Specified units were not recognized.\r\n");
			}
			WriteUserSettings();
		}
		RedirectResponse(sock, "advanced.htm");

	} else {
		NotFoundResponse(sock, url);
		iprintf("HTTP Post Form Processor: We did not match any page\r\n");
	}
	return 1;
}
void RegisterPost() {
   SetNewPostHandler(MyDoPost);
}

void DisplayFirmwareVersion(int sock, PCSTR url) {
   writestring(sock, AppName);
}
void DisplayIOCardVersion(int sock, PCSTR url) {
	char output[100];
	//siprintf(output, "%s", std::string(g_IOCardVersion).c_str());
	siprintf(output, "%s", g_IOCardVersion.c_str());
	writestring(sock, output);
}
void DisplayUnitsDropDown(int sock, PCSTR url) {
	siprintf(output, "<select name=\"units\">");
	if (g_USUnits) {
		siprintf(output+strlen(output), "<option value=\"metric\">Metric (Meters)</option>");
		siprintf(output+strlen(output), "<option value=\"us\" SELECTED>US (Feet)</option>");
	} else {
		siprintf(output+strlen(output), "<option value=\"metric\" SELECTED>Metric (Meters)</option>");
		siprintf(output+strlen(output), "<option value=\"us\">US (Feet)</option>");
	}
	siprintf(output+strlen(output), "</select>");
	writestring(sock, output);
}

void getAJAXSystemStatus(int sock, PCSTR url) {
	unsigned long secs = g_UptimeSeconds;
	unsigned int days = secs/86400; secs -= days*86400;
	unsigned int hours = secs/3600; secs -= hours*3600;
	unsigned int mins = secs/60; secs -= mins*60;

	siprintf(output, "<table border=1 cellspacing=0 cellpadding=5>");
	siprintf(output+strlen(output), "<tr><td colspan=3>System Uptime: %d Day", days);
	if (days != 1) siprintf(output+strlen(output), "s");
	siprintf(output+strlen(output), ", %d Hour", hours);
	if (hours != 1) siprintf(output+strlen(output), "s");
	siprintf(output+strlen(output), ", %d Minute", mins);
	if (mins != 1) siprintf(output+strlen(output), "s");
	siprintf(output+strlen(output), ", %d Second", secs);
	if (secs != 1) siprintf(output+strlen(output), "s");
	siprintf(output+strlen(output), "<br><br></td></tr>");

	siprintf(output+strlen(output), "<tr><td>Travel Direction:</td><td colspan=2");
	if (g_DirectionTTL == 0) {
		siprintf(output+strlen(output), " bgcolor=#FF6666>I/O Card Not Available");
	} else {
		if (g_DirectionForward) {
			siprintf(output+strlen(output), ">Forward");
		} else {
			siprintf(output+strlen(output), ">Reverse");
		}
	}
	siprintf(output+strlen(output), "</td></tr>");

	siprintf(output+strlen(output), "<tr><td>-</td><td><b>Receiver 1</b></td><td><b>Receiver 2</b></td></tr>");
	siprintf(output+strlen(output), "<tr><td><b>Receiving Data</b></td>");
	if (g_ReceiverTTL[0] == 0) {
			siprintf(output+strlen(output), "<td bgcolor=#FF6666>No</td>");
		} else {
			siprintf(output+strlen(output), "<td bgcolor=#00FF00>Yes</td>");
		}
	if (g_ReceiverTTL[1] == 0) {
			siprintf(output+strlen(output), "<td bgcolor=#FF6666>No</td>");
		} else {
			siprintf(output+strlen(output), "<td bgcolor=#00FF00>Yes</td>");
		}
	siprintf(output+strlen(output), "</tr>");
	writestring(sock, output);output[0] = '\0'; //Clear output buffer

	//siprintf(output+strlen(output), "<tr><td><b>GPS Solution Status</b></td><td>%s</td><td>%s</td></tr>", std::string(g_ReceiverFields[0][0]).c_str(), std::string(g_ReceiverFields[1][0]).c_str());
	//siprintf(output+strlen(output), "<tr><td><b>GPS Position Type</b></td><td>%s</td><td>%s</td></tr>", std::string(g_ReceiverFields[0][1]).c_str(), std::string(g_ReceiverFields[1][1]).c_str());
	//siprintf(output+strlen(output), "<tr><td><b>Satellites Tracked</b></td><td>%s</td><td>%s</td></tr>", std::string(g_ReceiverFields[0][9]).c_str(), std::string(g_ReceiverFields[1][9]).c_str());
	//siprintf(output+strlen(output), "<tr><td><b>Satellites Used</b></td><td>%s</td><td>%s</td></tr>", std::string(g_ReceiverFields[0][10]).c_str(), std::string(g_ReceiverFields[1][10]).c_str());
	siprintf(output+strlen(output), "<tr><td><b>GPS Solution Status</b></td><td>%s</td><td>%s</td></tr>", g_ReceiverFields[0][0].c_str(), g_ReceiverFields[1][0].c_str());
	siprintf(output+strlen(output), "<tr><td><b>GPS Position Type</b></td><td>%s</td><td>%s</td></tr>", g_ReceiverFields[0][1].c_str(), g_ReceiverFields[1][1].c_str());
	siprintf(output+strlen(output), "<tr><td><b>Satellites Tracked</b></td><td>%s</td><td>%s</td></tr>", g_ReceiverFields[0][9].c_str(), g_ReceiverFields[1][9].c_str());
	siprintf(output+strlen(output), "<tr><td><b>Satellites Used</b></td><td>%s</td><td>%s</td></tr>", g_ReceiverFields[0][10].c_str(), g_ReceiverFields[1][10].c_str());

	siprintf(output+strlen(output), "<tr><td><b>Vector Length</b></td><td>");
	if (g_USUnits) { //*3.28084
		double rf02 = atof(g_ReceiverFields[0][2].c_str());
		double rf12 = atof(g_ReceiverFields[0][2].c_str());
		sprintf(fnum, "%3.2f", rf02 * 3.28084);
		siprintf(output+strlen(output), "%s ft", fnum);
		sprintf(fnum, "%3.2f", rf12 * 3.28084);
		siprintf(output+strlen(output), "</td><td>%s ft", fnum);
	} else {
		//siprintf(output+strlen(output), "%s m</td><td>%s m", std::string(g_ReceiverFields[0][2]).c_str(), std::string(g_ReceiverFields[1][2]).c_str());
		siprintf(output+strlen(output), "%s m</td><td>%s m", g_ReceiverFields[0][2].c_str(), g_ReceiverFields[1][2].c_str());
	}
	siprintf(output+strlen(output), "</td></tr>");

	//siprintf(output+strlen(output), "<tr><td><b>Vector Heading</b></td><td>%s&#176;</td><td>%s&#176;</td></tr>", std::string(g_ReceiverFields[0][3]).c_str(), std::string(g_ReceiverFields[1][3]).c_str());
	//siprintf(output+strlen(output), "<tr><td><b>Vector Pitch</b></td><td>%s&#176;</td><td>%s&#176;</td></tr>", std::string(g_ReceiverFields[0][4]).c_str(), std::string(g_ReceiverFields[1][4]).c_str());
	//siprintf(output+strlen(output), "<tr><td><b>Heading Accuracy</b></td><td>+/- %s&#176;</td><td>+/- %s&#176;</td></tr>", std::string(g_ReceiverFields[0][6]).c_str(), std::string(g_ReceiverFields[1][6]).c_str());
	//siprintf(output+strlen(output), "<tr><td><b>Pitch Accuracy</b></td><td>+/- %s&#176;</td><td>+/- %s&#176;</td></tr>", std::string(g_ReceiverFields[0][7]).c_str(), std::string(g_ReceiverFields[1][7]).c_str());
	siprintf(output+strlen(output), "<tr><td><b>Vector Heading</b></td><td>%s&#176;</td><td>%s&#176;</td></tr>", g_ReceiverFields[0][3].c_str(), g_ReceiverFields[1][3].c_str());
	siprintf(output+strlen(output), "<tr><td><b>Vector Pitch</b></td><td>%s&#176;</td><td>%s&#176;</td></tr>", g_ReceiverFields[0][4].c_str(), g_ReceiverFields[1][4].c_str());
	siprintf(output+strlen(output), "<tr><td><b>Heading Accuracy</b></td><td>+/- %s&#176;</td><td>+/- %s&#176;</td></tr>", g_ReceiverFields[0][6].c_str(), g_ReceiverFields[1][6].c_str());
	siprintf(output+strlen(output), "<tr><td><b>Pitch Accuracy</b></td><td>+/- %s&#176;</td><td>+/- %s&#176;</td></tr>", g_ReceiverFields[0][7].c_str(), g_ReceiverFields[1][7].c_str());
	writestring(sock, output);output[0] = '\0'; //Clear output buffer

	siprintf(output+strlen(output), "<tr><td><BR><BR><b>Current Distance</b></td><td><BR><BR>");
	if (g_USUnits) {
		sprintf(fnum, "%3.2f", g_ReceiverDistance[0][0] * 3.28084);
		siprintf(output+strlen(output), "%s ft", fnum);
	} else {
		sprintf(fnum, "%3.2f", g_ReceiverDistance[0][0]);
		siprintf(output+strlen(output), "%s m", fnum);
	}
	siprintf(output+strlen(output), "</td><td><BR><BR>");
	if (g_USUnits) {
		sprintf(fnum, "%3.2f", g_ReceiverDistance[1][0] * 3.28084);
		siprintf(output+strlen(output), "%s ft", fnum);
	} else {
		sprintf(fnum, "%3.2f", g_ReceiverDistance[1][0]);
		siprintf(output+strlen(output), "%s m", fnum);
	}
	siprintf(output+strlen(output), "</td></tr>");

	siprintf(output+strlen(output), "<tr><td><b>Offline Distance</b></td><td");
	double dbl_od = fabs(g_ReceiverDistance[0][1]);
	if (dbl_od > 0.5) {
		if (dbl_od > 1) { //Red
			siprintf(output+strlen(output), " bgcolor=#FF6666");
		} else { //Yellow
			siprintf(output+strlen(output), " bgcolor=#FFFF00");
		}
	}
	if (g_USUnits) {
		dbl_od *= 3.28084 * 12; //to feet, to inches
		sprintf(fnum, "%3.1f", dbl_od);
		siprintf(output+strlen(output), ">%s in", fnum);
	} else {
		sprintf(fnum, "%3.2f", dbl_od);
		siprintf(output+strlen(output), ">%s m", fnum);
	}
	siprintf(output+strlen(output), "</td><td");

	dbl_od = fabs(g_ReceiverDistance[1][1]);
	if (dbl_od > 0.5) {
		if (dbl_od > 1) { //Red
			siprintf(output+strlen(output), " bgcolor=#FF6666");
		} else { //Yellow
			siprintf(output+strlen(output), " bgcolor=#FFFF00");
		}
	}

	if (g_USUnits) {
		dbl_od *= 3.28084 * 12; //to feet, to inches
		sprintf(fnum, "%3.1f", dbl_od);
		siprintf(output+strlen(output), ">%s in", fnum);
	} else {
		sprintf(fnum, "%3.2f", dbl_od);
		siprintf(output+strlen(output), ">%s m", fnum);
	}
	siprintf(output+strlen(output), "</td></tr>");

	if (g_MaxOfflineDistanceOverrideActive) {
		siprintf(output+strlen(output), "<tr><td colspan=3 bgcolor=#FFFF00><b>Override of Maximum Offline Distance is Enabled</b></td><tr>");
	}

	siprintf(output+strlen(output), "</table>");
	writestring(sock, output);
}

void getAJAXPathList(int sock, PCSTR url) {
	if (g_WayPointCount == 0) {
		siprintf(output, "There are no waypoints defined.<BR><BR>");
	} else {
		siprintf(output, "<table border=1 cellspacing=0 cellpadding=5>");
		siprintf(output+strlen(output), "<tr><td>Heading<BR>(in Degrees)</td><td>Distance<BR>(in ");
		if (g_USUnits) {
			siprintf(output+strlen(output), "Feet");
		} else {
			siprintf(output+strlen(output), "Meters");
		}
		siprintf(output+strlen(output), ")</td><td>Path</td></tr>");

		if (g_WayPointCount > 0) {
			for (int i = 0; i < g_WayPointCount; i++) { //-1
				if (i > 0 && i == g_ActiveWayPointSegment) { //Add the row showing current location
					siprintf(output+strlen(output), "<tr><td bgcolor=#CCCCCC><b>");
					sprintf(fnum, "%3.4f", g_ActiveWayPointHeading);
					siprintf(output+strlen(output), "%s", fnum);
					siprintf(output+strlen(output), "</b></td><td bgcolor=#CCCCCC><b>");
					if (g_USUnits) {
						sprintf(fnum, "%3.2f", g_ReceiverDistance[1][0] * 3.28084);
					} else {
						sprintf(fnum, "%3.2f", g_ReceiverDistance[1][0]);
					}
					siprintf(output+strlen(output), "%s", fnum);
					siprintf(output+strlen(output), "</b></td><td bgcolor=#CCCCCC><b>Current Location");
					siprintf(output+strlen(output), "</b></td></tr>");
				}

				siprintf(output+strlen(output), "<tr><td>");
				sprintf(fnum, "%3.4f", g_WayPointHeading[i]);
				siprintf(output+strlen(output), "%s", fnum);
				siprintf(output+strlen(output), "</td><td>");
				if (g_USUnits) {
					sprintf(fnum, "%3.2f", g_WayPointDistance[i] * 3.28084);
				} else {
					sprintf(fnum, "%3.2f", g_WayPointDistance[i]);
				}
				siprintf(output+strlen(output), "%s", fnum);
				siprintf(output+strlen(output), "</td><td>");
				if (g_WayPointStraight[i]) {
					siprintf(output+strlen(output), "Straight");
				} else {
					siprintf(output+strlen(output), "Arc");
				}
				siprintf(output+strlen(output), "</td></tr>");
				writestring(sock, output);output[0] = '\0'; //Clear output buffer
			}
		}
		siprintf(output+strlen(output), "</table>");
	}
	writestring(sock, output);
}

That call creates a table that looks like the image below. As you can see, current up time is 38 hours, no traps since I removed the std::string() stuff.

[Ridgeglider]

One easy thing to do would be to always use sniprintf() instead of siprint() to guarnatee against buffer overruns. You're calling siprintf a LOT with formats that produce somewhat unpredictable output lengths.

[tod]

At a minimum add a check at the end to see what the length of output is. If you're anywhere close to 2000, I would increase its size. If it ever reports back greater than 2000 you should assert. Letting the buffer overrun makes for a very hard to find trap. Using a "safe" version of sprintf as Ridgeglider suggests is even better but requires more work. It's also only "safe" from memory overrun it's still not type safe. The code as written is begging you to write an AppendString() method that does the indexing into output, checks that the new string will fit and then if it does adds it and returns true. Since you use floats I would just use sprintf, toggling between siprintf and sprintf strikes me as prone to error and most likely a premature optimization.

Since you've already paid the price for streams (by including <iostream>) I would suggest using ostringstream to do all this (the wiki has a brief tutorial). It would be much cleaner, immune to memory overflow and typesafe. Right now you have a maintenance issue when you change the type of any variable from integer to float/double or vice versa.

For future reference, you might find it way more fun (I know I do) to put all the HTML in the HTML page and then just have getAJAXSystemStatus create a JSON literal object and pass it back to the HTML page. The ajax responder on the browser could then just stick all the appropriate values in all the appropriate tags. That would get you a better separation of concerns, require less NB memory and be faster since much less data is sent over the line. It should also be easier to write and maintain. Using JQuery makes it pretty easy to do. Using knockout data binding makes it trivial, but there is a pesky learning curve.