All Stories at gladfelter.net/weblog

C is for Cookie!

Fri, 28 Apr 2006 12:38:00 -0700

A couple of spoofs of the creepy V for Vendetta Trailer

BoingBoing found a hilarious ripoff of the V for Vendetta Trailer. A friend of mine just made his own ripoff for the software product he works on, Agilent VEE Pro. Take a look at it here.

Discouraging Comment and Trackback Spam on Geeklog

Tue, 25 Apr 2006 14:02:00 -0700

My blog is barely visible in the universe and I've already started getting trackback spam. Read on to see what I've done to discourage it.

Google and other search engines obey an attribute on links that causes them to ignore the links and not give any "juice" to the websites that are linked to. Comment and trackback spammers try to use your website to give themselves more credibility by putting links to their sites from yours through those comments and trackbacks. Using the "nofollow" attribute prevents this from working. You can still have people posting bogus trackbacks, but at least now they don't gain anything by it.

To implement this in Geeklog v 1.4rc2 I made these changes:

In lib-trackback.php I changed these lines:} else {	$template->set_var ('trackback_excerpt', $excerpt);	$template->set_var ('trackback_excerpt_readmore',	$excerpt . ' ' . $readmore);	$template->set_var ('excerpt_br', '');}to these lines by adding one line:} else {	$excerpt = ereg_replace("<s*a[^>]+href=", "<a rel='nofollow' href=", $excerpt);	$template->set_var ('trackback_excerpt', $excerpt);	$template->set_var ('trackback_excerpt_readmore',	$excerpt . ' ' . $readmore);	$template->set_var ('excerpt_br', '<br>');}

In lib-comment.php I changed these lines:	$A['comment'] = str_replace( '$', '$', $A['comment'] );	$A['comment'] = str_replace( '{', '{', $A['comment'] );	$A['comment'] = str_replace( '}', '}', $A['comment'] );	// Replace any plugin autolink tags	$A['comment'] = PLG_replaceTags( $A['comment'] );to these by adding one line:	$A['comment'] = str_replace( '$', '$', $A['comment'] );	$A['comment'] = str_replace( '{', '{', $A['comment'] );	$A['comment'] = str_replace( '}', '}', $A['comment'] );	$A['comment'] = ereg_replace("<s*a[^>]+href=", "<a rel='nofollow' 	href=", $A['comment']);	// Replace any plugin autolink tags	$A['comment'] = PLG_replaceTags( $A['comment'] );

In formattedcomment.thtml I changed this line:

<a href="{trackback_url}">{trackback_title}</a> {trackback_from_blog_name}

to this line:

<a rel='nofollow' href="{trackback_url}">{trackback_title}</a> {trackback_from_blog_name}

The rel="nofollow" line tells Google, etc, to not use the link. This is only turned on for comments and trackbacks. Any articles I post will still be followed by search engines, should they ever care to visit!

Regular Expression for today

Tue, 25 Apr 2006 11:32:34 -0700

The regular expression to select a multi-line C++ comment in Visual Studio: /*((*~(/))|[^*]|n)**/ Fun!

Watch out for the AssemblyCultureAttribute for assemblies

Fri, 21 Apr 2006 13:50:00 -0700

I spent about 2 days on a bug recently. The bug was a result of a major architectural change I had to make to an application as a result of a requirements bug in an implementation of a feature by a previous developer. Read on to find out an interesting feature (ahem, attribute) of assemblies.

The Original Problem

A previous developer had created an assembly that would reflect on arbitrary other assemblies and return essential information so that a user interface could allow users to select functions from those assemblies. Unfortunately, the developer hadn't taken into account that you cannot unload assemblies from an appdomain once they're loaded. This means that the program is stuck locking any assemblies that are loaded to be viewed in the UI.

AppDomains

This was inconvenient and confusing: users couldn't move or recompile assemblies without shutting down our application and attempts to load a different version of the assembly (at a different file location, for example) would have inconsistent, unexpected behavior. The answer according to Microsoft is to use an AppDomain.

AppDomains are to .NET what processes are to unmanaged code: a safe, contained area that must use inter-domain (process) communication techniques to communicate outside of itself. .NET allows you to have mutliple AppDomains in the same unmanaged process, and you can create and delete domains at will (aside from the default, main application domain.) You can therefore remove any references a process has to an assembly by unloading the appdomain that references the assembly.

The Solution

My solution was to load an instance of the class that will do the reflecting in the separate appdomain, which means I need to locate my assembly and the class inside of it and call AppDomain.CreateInstance with that information. Here is a simplified diagram of the solution:Figure 1

I knew this could work because I had already implemented it for the runtime aspect of the application. This design time aspect should in theory be easier because I don't need to marshal the actual call parameters with various arbitrary types from one appdomain to another, just the descriptions of the classes, methods, and parameters in the classes I reflect on.

Reviewing the reflection classes that had already been implemented, I found they could be made to work across appdomains with only a couple of minor changes. They needed to be derived from MarshalByRefObject rather than from Object so that the .NET infrastructure would not try to marshal them by value across the appdomain. This was important because the reflection class loads the other classes and assemblies it is reflecting on, so that if it does its work in our appdomain, then we are stuck with those assemblies in our appdomain. By keeping the reflector class in a separate appdomain, implementing and using only object types that are defined by reflector.dll or mscorlib.dll, and using marshal-by-ref we can be sure we never directly reference any of the assemblies under reflection.

A Two-Day Bug

Unfortunately this solution generated an exception when I tried to create my own reflector class in a separate AppDomain. Here's the code:Figure 2

The error occurs on the cast from Object to AssemblyInspector. The error says that it cannot locate the assembly:
An unhandled exception of type 'System.IO.FileNotFoundException' occurred in Unknown Module. Additional information: File or assembly name Reflection, or one of its dependencies, was not found.

Looking into the call stack I saw that it was trying to find the assembly:

"Reflection, Version=1.0.3.5, Culture=en-US, PublicKeyToken=ba91c806523fc1cf" (I changed the version and the public key token, but everything had some non-blank value.)

If I used CreateInstance instead of CreateInstanceFrom with the Assembly.Fullname of the assembly, I would get the same error on CreateInstance. So what was going on?

Damned Culture

It turns out the important part of this was the "Culture=en-US". The culture is specified by placing a value in AssemblyInfo.cs like this:

[assembly: AssemblyCulture("en-US")]

The previous developer had put that in there. Unfortunately, using that value on CreateInstance (or when trying to load other types in which case apparently .NET tries to do something equivalent, as during the type cast) tells the .NET loader that the assembly is a resource assembly and it must be found via the resource loading rules. It will look in various subdirectories of the main assembly for the file such as .en-US, .reflection, .reflectionen-US, but it will ignore the assembly in the main directory.

This would even happen if you tried to directly reference an assembly with a non-blank culture. The reason this didn't happen with reflection.dll when the client created an instance of the reflector class factory class (and previously, the reflector class itself) is that it was always created and loaded via COM interop, which apparently doesn't use the resource loading rules when finding the assembly (it uses CreateInstanceFrom, I guess.)

Once I removed the culture attribute from the assembly everything worked great.

Lessons Learned

I spent too much time on this stupid bug. I fell into the trap of making up too many theories and eliminating them rather than eliminating each line of code that could be the cause of the problem. Keith Hill recommended I start from scratch with a test project, and I found the problem in about half an hour as I built the project up to resemble the DLL that was giving me hell.

In my defense, there is a ton of opaque technology in this assembly: assembly signing, reflection, cross-app-domain interop, security, etc. That said, in my experience 95% of bugs are much simpler than they appear and simple test & elimination will do once the problem area has been identified. Next time I'll try to get to basics and not start diving into .NET cross-app-domain security implementation to find my solution!

Why Serial and Binary don't mix

Thu, 16 Feb 2006 16:11:00 -0700

RS-232 and other simple serial protocols have in the past been the backbone of instrument-controller communications. Until a few years ago if you wanted to talk to an instrument or device, you were either using GPIB or serial. Click below to read why serial has proved to be a headache for so many people for so long.

Amost all instrument control communication is logically serial communication at some level. Since you connect a PC to an instrument via a cable that has either a couple or perhaps 8 data lines (in the case of GPIB), you don't have random access to the instrument, so you have to rely on messages that send commands or queries to the instruments, and messages to send and receive data in a flat, serial format to and from the instrument.

VXI (and PXI?) is the only communication mechanism that I'm aware of that presented a random access view of instruments to the controller, which means you're dealing with "dumb" instruments. I say dumb because one tenet of modern software design is information-hiding. If you don't have to know the inner details of every system you touch you can work at a higher level and get your immediate job done faster and easier. Twidlling bits on an instrument across the VXI backplane is the opposite of information hiding. An instrument that exposes its guts like that can't change or do any software "work" for you because it would get in the way of this random access to the instrument. Anyway, VXI is for another day

So what makes serial hard? Two things: data formats and a deterministic way to tell a transmission is complete.

Data formats refer to how to serialize data from a complex, binary representation in the PC or instrument, to a flat, serial representation that can go across the wire. This is alternately called marshalling. In the instrumentation world the answer has always been to create formats that are human readable and intuitive. So a IEEE Standard 754 binary representation of a particular floating point number in the memory of a PC is changed into the ASCII text "1.049e-3" for transmission over the wire. These textual representations of datatypes common in T&M were standardized into IEEE Standard 488.2.

One thing missing from most 488.2 representations is framing. Framing means putting information at the beginning of a serialized value saying how much data to expect in the value. If you always know how much to read, you know when to stop reading. There are some 488.2 binary types that have framing, they are called definite-length 488.2 binary blocks.

The language of commands and queries used by most T&M instrumentation is called SCPI (Standard Commands for Programmable Instruments). For example, to ask a DMM (Digital Multimeter) what voltage it is currently reading, you would send it "MEAS:VOLTS?". SCPI also has no framing for its commands.

One thing SCPI does have is a defined read termination character. Whenever a SCPI device reads this character, it knows the current message is complete. The character is the ASCI linefeed, code 10. This is not too unexpected, since old line printers would receive a Carriage return code followed by a linefeed code to start a new line of text. Trivia: it takes longer for the printer head carriage to move all the way back to the beginning of a line than it does for the paper roll to advance one line, so CR, ASCII code 12, was always sent first.

GPIB (formerlly HP-IB for Hewlett-Packard Interface Bus and standardized as IEEE 488) has an alternate solution to this problem. Anyone who's bought a GPIB cable knows how thick (and expensive) they are. One of the things you're paying for is a line devoted to telling you to stop reading data. There is an EOI transmission line, that when raised during the transmission of a byte, indicates that that byte is the last byte in the message, so you can stop reading. On well-formed SCPI over GPIB, the EOI line is always raised when the Linefeed character is sent to indicate the end of the message. Your GPIB I/O librarys (VISA, NI-488, SICL, or what have you) Read() method will return, and you'll have a well-formed message to read.

So why is serial so hard? If you want to send binary data over serial, assuming the data is random about every 255th character will be the ASCII linefeed character. That explains the existence of the 488.2 Definite-length binary block with its built-in framing. Someone reading from a 488.2-compatible data stream would see the # character and if the following character was a numeral, they would know it was a definite length block with the length information prepended before the start of the data payload. There are I/O libraries, such as VISA (Virtual Instrument Software Architecture) that have methods for interpreting such blocks. If you get it wrong and read too much, you'll either time out and your read function will return an error or your program will hang indefinitely.

Getting all this right can be a real pain, and sometimes there's no hope. A team I was on had to deal with an instrument that was designed to send a waveform image to a GPIB printer. It would format the data as a raw binary block that the printer expected. Apparently those that designed the printer data format decided that since a printer has nothing to do when it isn't reading data, there is no reason to tell it when the data is complete. :(

There was no other way to get the nice waveform image out of the instrument to give to the user. Users could choose to hook up their PC to the instrument via a serial cable, and it is reasonable to expect the same behavior of our application on any supported I/O method. We settled on an intra-character timeout. We read one byte at a type from the incoming I/O stream, and tell the I/O library we were using to stop reading and report an error if more than, say, a quarter-second went by without receiving data. This meant that the operation to display the waveform to the user would always take a quarter-second longer than it needed to.

But... we found in testing that sometimes the instrument would burp. It would process data to generate the waveform as it went, and it would sometimes pause as it was processing longer than we expected. I think in the end we had to settle for a one-second intracharacter timeout, which was a far from ideal solution.

A problem specific to older serial devices is handshaking. Handshaking is serial's way of traffic coordination. A serial cable would typically have 4 or more transmission lines. One for data up, one for data down, and the other two to coordinate those data lines up and down. If a receiving device kept a handshaking line high, the talking device would continue to talk. If the receiving device set the line to low, the talking device would stop talking. This allowed both sides to absorb data at a rate they could handle so that no bits would be lost.

On another project, we had an instrument that was sending us malformed data on some computers but not others. We got the bad data during line-by-line debug, but not during run debugging. We spent several weeks looking into NT's serial implementation, the source code behind the I/O libraries we were using, and probably a dozen different places. Can you guess what the problem was?

The instrument had a broken handshaking implementation. Older instruments sometimes used their own implementation of RS-232 rather than the defacto-standard RS-232 chipset to save a couple of bucks (back when they cost that much.) This particular instrument had a flawed handshaking implementation that would rear its ugly head when the PC couldn't keep up with the instrument, making the data look malformed. It may be a truism that more bad things happen when people try to save a couple of bucks than when malice is intended.

Today serial is being killed by USB for higher-end devices, but it is still a common way of controlling devices under functional test and for communication between embedded devices. Many T&M instruments support raw TCP/IP socket communication, which can still have the read termination problem, but thankfully not the handshaking problem of days past.

Perhaps I could have boiled this essay down to something simpler

Serial Considered Harmful

Getting the Geeklog Weblog host up and running on GoDaddy

Thu, 02 Feb 2006 22:50:00 -0700

Click below to see a brief overview of my experience getting the Geeklog weblog hosting software up and running on my Godaddy PHP and MySQL-enabled account.

Godaddy does not support telnet or SSH into its servers, so after downloading the latest Geeklog revision (a release candidate 1.4.0rc2) I had to get it on the server via FTP. FTP also allows for changing file and directory attributes as the Geeklog install instructions require.

I eventually decided to put all the directories except the public_html directory under /weblog_engine. I put the public_html directory under /weblog so that users can access my weblog as gladfelter.net/weblog. I made the weblog_engine directory invisible to any snoopers by creating a file called .htaccess in the /weblog_engine directory and putting this in the file:

Deny from all

This causes Apache to deny all access to this directory so that snoopers can't read my database passwords, etc. The database was a pain to configure, after creating a database using GoDaddy's configuration manager I didn't see the actual server for my database was mysql99.secureserver.net. The instructions on the GoDaddy site use mysql.secureserver.net and I didn't realize that was a suggestion. It didn't help that I didn't have a lot of confidence in the GoDaddy database, as it wasn't immediately clear to me that the database name and the user name for the database will always be the same.

After successfully configuring config.php, etc there was still a problem with submitting comments. I discovered on the geeklog site that some hosting services do not allow a database lock command. The command in question is "LOCK TABLES" and its counterpart "UNLOCK TABLES". This is to prevent race conditions, I guess, but I don't anticipate a problem with this small site, so I removed the commands as recommended in the geeklog user forums.

I've tweaked some display stuff since then. Overall, I'm really impressed with the geeklog software!

Blog Online

Wed, 01 Feb 2006 17:01:18 -0700

Welcome to my glorius blog I'm going where many have gone before.