• Lexical Analysis – convert the input text into a list of tokens that distinguishes language features from whitespace, comments, pre-processor commands, etc. (“Unknown character on line 2″ errors are produced by this stage).
• Syntax Analysis – convert the list of tokens into a syntax tree which shows the entire file hierarchically as perfectly valid expressions (this is where “Missing semicolon on line 4″ type errors come from).
• Semantic Analysis – process the syntax tree to make sure that it makes sense according to the rules of the language. Type checking falls under this category for instance, so this is where “warning, assigning ‘int’ to ’short’ – possible loss of data” type errors come from.

SimpleHLST only needs a very basic language and I’m not planning to have it support complex data, so I don’t think we’ll need a semantic analysis stage at the moment.

An example

Suppose we have the following mathematical expression, f(t) = 2t + k, and we want to implement it in some program; we could write something like

f(t) = 2*t + k.

Before any more advanced stages can take place, the compiler must extract the information from this expression. This is exactly the same principle as we humans extracting information from text. We have languages that each have a vocabulary of words with specific definitions which we can use to convey detailed information. Each word only represents a small piece of information; the context and sentence structure contains the rest.

Programming languages are exactly the same and this is where Lexical Analysis (or, colloquially, Tokenisation) gets its name from. Programming languages tend to use symbols, or tokens, instead of words for brevity. The output of this stage would be an array that looks somewhat like the following table.

Now the parser can operate on this set of tokens instead of having to wade through the text itself. This example is, obviously, very simplistic and having a separate tokenising and parsing stages may even complicate things, but it is incredibly useful for handling more complex languages such as C and Verilog.

Finite Automata

It’s very tempting to try and define all the jobs of the parser as regular expressions (regex) to begin with, but there’s a very commonly used construct that they cannot cope with: parenthesis. It’s easy to define a regex that can handle say “()” but that can’t adapt to “(())” and so on. We need something more general.

Finite Automata (FA) are abstract state machines that operate on sets of data. Despite being very abstract we can actually use regular expression to define a simple FA but we may not be able to do the reverse of this. Let’s consider a compiler reading in a keyword or a variable name.

This diagram shows a Deterministic Finite Automata (DFA) that will transition into the terminal state when it encounters a letter and will stay in that state for every alphanumeric character or underscore it encounters thereafter. Every state transition is clearly defined therefore it is said to be deterministic but this may not always be the case.

One of the state transitions in this diagram is marked with ε which means that it is taken if no other transitions match. This allows us to make very complex state transitions with optional states, possibly even bypassing large sections of the state diagram (much like the * and ? operators allow complex regular expressions). Unfortunately every state transition is no longer rigidly defined and hence they are called Non-deterministic Finite Automata (NFA).

ε-edges should be used with caution. It is perfectly acceptable (at least, in the theory) to have several ε-edges transitioning out of a single state but this means that all of these edges must be taken at once. The first deterministic edge taken on any of these branches then defines the next state of the NFA hence all other edges must be discarded. In practice this makes NFAs unattractive for compiler front-ends because they potentially need large amounts of memory and computation. It is possible to convert NFAs into DFAs but I think I’ve rambled on long enough about the theory now.

From Theory to Code

Theory is useful and worth understanding but it can sometimes be difficult to map onto actual code. I learned a lot from Tommy Carlier’s blog series before I wrote my own lexer so it might be worth looking there as well to fill in any gaps I leave. There are several ways to implement finite automata in code and the final decision will probably depend on the language you’re using. I’ve decided to write SimpleHLST in C++ which might not be the best decision but it’s the language I feel most comfortable with.

output = 15*in_1 + 39*in_2

If we look at the above expression there are only actually three types of token (words, integers and symbols) so the finite automaton only needs three states. The SimpleHLST tokeniser implements each state as a function call inside the main worker function, readNextToken, as shown below.

int Tokeniser::readNextToken ()
{
    // Skip over all whitespace
    munchWhitespace ();
 
    // Make sure we don't advance beyond the end of file
    if (atEnd ())
        m_curTokType = T_EOF;
    else if (isalpha (m_char) || m_char == '_')
        readWordToken ();
    else if (isdigit (m_char))
        readNumericalToken ();
    else // Assume it must be a symbol
        readSymbolToken ();
 
    // Return the token type
    return m_curTokType;
}

Before attempting to read a token, this function skips over all whitespace using the function shown below. The obvious improvement this function needs is to be able to skip over comments.

void Tokeniser::munchWhitespace ()
{
    // Just keep reading until there's no whitespace left.
    while (isspace (m_char) && !m_eof)
        readNextChar ();
}

Finally, readWordToken is listed below which is capable of extracting “output” and “in_1″ from the expression given earlier. The pushBuffer and extractBuffer functions operate on a std::string member variable that is used to store words and integers. An accessor function, tokenData, can then be used to find out what the token data is.

void Tokeniser::readWordToken ()
{
    // Keep reading alphanumeric characters into the buffer.
    while (isalnum (m_char) || m_char == '_')
        pushBuffer ();
 
    // Store the token details
    extractBuffer ();
    m_curTokType = T_WORD;
}

You can download the source code for the tokeniser in the form of a Visual Studio 2010 solution. I haven’t used any features particular to VS2010 so it should be possible to build with any prior version or g++ but I can’t confirm this. I’ve also included some sample code that I hope will become the HDL we’ll feed into SimpleHLST at a later stage.

To try it out: build the project and open a command prompt in the same directory as the SimpleHLST executable. Then type SimpleHLST <expression or filename> on the command line to see the output. The picture above shows some sample output.

T_EOF

Now we can read a stream in terms of tokens instead of characters we can get started on the parser! Next time, of course…

I’m not sure how long it’ll take me to complete so I’ve decided to split my progress over multiple blog posts. I’ll start by briefly describing what high level synthesis actually is and what elements I’m expecting my simple program to have.

Digital Design Flow

Chip design is immensely difficult because we can fit so many transistors in them (around 30 million on a pin head!) so we can’t think of it as a single design. It is broken up into many smaller modules that are designed under various “levels of abstraction” which basically means a set of interdependent tasks.

The figure above shows the process used for digital chip design and it may be surprising just how far removed the physical transistors are. There are so many in modern designs that it just becomes impractical to consider each one. Hardware Description Languages (like Verilog, VHDL and SystemC) are used for every level apart from the physical layout and look very similar to programming languages but there is a difference that will become apparent later.

You may be wondering if this means that modern chips are actually designed by programs. Well they certainly play a very large part in a lot of widespread designs. Hardware description language (HDL) code is fed into compilers in exactly the same way that programming language code is, but there is a specialist compiler for each dashed line in the figure. High level synthesis tools produce RTL, RTL compilers produce netlists, and layout tools produce silicon layouts. It does take care and skill to drive these tools effectively and often some hand tweaking may be required to make the most of the design, but ultimately this is how it all works.

I’m going to focus on the thing at the very top. I’m planning to make a very basic HDL that is capable of describing equations and algorithms, and then feed that into SimpleHLST which will generate Verilog code. In an ideal situation I would synthesise the output of this onto an FPGA but that is a long way off just yet.

The Plan

There is a lot to consider for this project, and I think the tasks will break down as follows.

• Read in language and generate data flow graph (DFG)
• Run ASAP/ALAP algorithms on the DFG to work out what functional units are required
• Use constrained algorithms on the DFG to get an optimal design for our purpose
• Join the functional units together using data-path synthesis techniques
• Finally, synthesise a controller to drive the data-path

After this lengthy process SimpleHLST should produce some Verilog code that we can simulate to make sure it works.

Onwards!

In the next part we’ll design a very basic HDL that will fit our needs and, hopefully, start a program that may even understand it!

One of my minor duties on this dual-job is to assemble slides from about twelve people into a large presentation, with cover slides for each speaker, every Friday for a progress meeting we all have. Naturally the first Friday I just did it by hand by importing each one in turn into PowerPoint. However it is a fairly tedious job, and to paraphrase a certain member of staff: why do something by hand when I have a powerful computer under the desk?

So I began to investigate automating the process.

Turns out that Python has an IMAP module in its standard library, which isn’t too surprising I suppose as the Python standard library is enormous. After some playing I managed to write a program that logged into my university email account and downloaded the appropriate PowerPoint attachments.

from imaplib import IMAP4_SSL
from email import message_from_string
from datetime import datetime, timedelta
import re
 
M = IMAP4_SSL ('imapserver')
M.login ('user', 'pass')
M.select ()
 
since = (datetime.today () - timedelta (days=5)).strftime ("%d-%b-%Y")
 
for user in users:
    typ, data = M.search (None, '(HEADER FROM "%s" SINCE %s)' % (user, since))
 
    for id in data[0].split ():
        typ, data = M.fetch (id, 'RFC822')
        msg = message_from_string (data[0][1])
        if not msg.is_multipart ():
            continue
 
        for part in msg.walk ():
            fn  = part.get_filename ()
            ext = re.match (r'.*\.(pptx?)$', fn):
            if not ext
                continue
 
            fp = file ('%s.%s.temp' % (user, ext.group (1)), 'w')
            fp.write (part.get_payload ())
            fp.close ()
 
M.close ()

This code would download any .ppt or .pptx attachment from anyone with an email address in the users list and save the file as <user>.ppt.temp or <user>.pptx.temp. It should be possible to decode the message part with msg.get_payload (decode=True) but it seemed to introduce a fair number of errors into the file. I think this is because it seems to convert line-by-line instead of as one large block. So I used the following code to fix this.

raw      = part.get_payload ()
raw      = re.sub (r'[\r\n]+', '', raw)
encoding = part['Content-Transfer-Encoding'].lower ()
if encoding == 'base64':
    raw = b64decode (raw)
elif encoding == 'quoted-printable':
    raw = quopri.decodestring (raw)
else:
    print "ERROR: Unknown coding strategy - ignoring."
    continue

Not particularly pretty, but it does the job.

So now I had all the emails downloaded, but I still needed to merge them all into a single file. At first I was thinking of making a C++ program to exploit the PowerPoint COM interface, but then I found Python for Windows Extensions which fully supports COM!

pythoncom.CoInitializeEx (pythoncom.COINIT_APARTMENTTHREADED)
gencache.EnsureModule ('{2DF8D04C-5BFA-101B-BDE5-00AA0044DE52}', 0, 2, 4)
gencache.EnsureDispatch ("PowerPoint.Application.12")
 
# Create an instance of PowerPoint and presentation
pp = Dispatch ("PowerPoint.Application.12")
pp.Activate ()
pres = pp.Presentations.Add ()
 
# Insert all the downloaded presentations
count = 1
for filename in os.listdir ('presentations'):
    pres.Slides.InsertFromFile (os.path.realpath (filename), count)
    count = pres.Slides.Count
 
# Save and exit
pres.SaveAs ('compiled.pptx')
pres.Close ()
pp.Quit ()
pythoncom.CoUninitialize ()

I was quite shocked at how straightforward it was to automate PowerPoint from Python, but this solution wasn’t quite good enough yet. Using InsertFromFile means that the imported presentation acquires the formatting of pres which is not what I wanted. Also, there appears to be a bug in PowerPoint 2007 which causes image references to be broken when importing from a .pptx into a .pptx with the COM interface.

Searching for a solution to the import with formatting issue lead me to this awesome VBA function which has been referenced many, many times. I ported this to Python and it worked perfectly! There was still the weird image problem, but I used a really crude fix for that:

for filename in [f for f in os.listdir ('presentations') if f[-4:] == 'pptx']:
    pres = pp.Presentations.Open (path.realpath (filename))
    pres.SaveAs (path.realpath (filename[:-1]))
    pres.Close ()

Yes. I converted all the .pptx’s into .ppt’s. Nothing intelligent here. Finally, I wrote a function to replace all the fonts added by the import with Arial. Job done.

fonts = {}
for i in range (1, pres.Fonts.Count + 1):
    fonts[pres.Fonts.Item (i).Name] = 1
fonts = fonts.keys ()
fonts.remove (u'Arial')
for font in fonts:
    pres.Fonts.Replace (font, u'Arial')

So after all that I have a Python program that logs into my email, downloads a load of PowerPoint files, converts them all to .ppt format, inserts them into a blank presentation, and then normalises the font to Arial. Not bad for a few hundred lines of code!

I did change the program a bit so that it would copy a template with title slides in and add the presentations to that instead of a blank file, and then set the date appropriately on the main title slide. But the key point here is that I’ve replaced my tedious Friday-morning activity with a single command: makepres.

Victory.

Yesterday I managed to download the front page of my website using libcurl. As good as that was as a learning experience, it wasn’t interesting or useful in the slightest. Today however, I decided to see if I could fetch my status updates from Twitter and display them in a program. So I had a look at the API documentation and it looks quite easy to use, with the exception of OAuth which I’m yet to get my head around. Thankfully, for now, basic authentication is still supported.

The Twitter API uses the REST (REpresentational State Transfer) paradigm which means there’s no concept of a state on the server; i.e. each transaction is considered separately. It also means that it uses HTTP, which is pretty simple to understand. Basically in a REST protocol the URI’s are objects in the system, and the HTTP verbs are how you interact with them. So a GET on a http://server/article?name=REST object would download an article named REST. Simple eh? Check this article if you’re interested.

Anyway, onto the meat ‘n’ taters. Data in a REST transaction is typically stored as XML or JSON. I considered downloading LibYAML and taking the JSON route but a) I already had Xerces, b) I understand XML more than JSON, and c) I couldn’t be bothered to learn yet another new thing.

Xerces is incredibly well written. If you look at the class listings of Xerces or Xalan you’ll appreciate they’re both enormous and support basically everything. In fact, right out of the box Xerces supports fetching XML documents over the internet using HTTP GET. I chose not to use this purely because I wanted to use libcurl. Thankfully libcurl is surprisingly easy to use:

static size_t _CurlWriteCB (void* ptr, size_t nLen, size_t cbElem,
                            CMemFile* pFile)
{
    size_t cbSizeAtStart;
    size_t cbSizeAtEnd;
 
    // Write data to file, but measure buffer size before and after.
    cbSizeAtStart = (size_t)pFile->GetLength ();
    pFile->Write (ptr, (UINT)(nLen * cbElem));
    cbSizeAtEnd   = (size_t)pFile->GetLength ();
 
    // Return the difference in buffer size, i.e. number of bytes written.
    return (cbSizeAtEnd - cbSizeAtStart);
}
 
BYTE* GetStatusesFromTwitter (char* szUserName, UINT& uiSize)
{
    // Attempt to initialise curl
    CURL* curl = curl_easy_init ();
    if (curl != NULL) {
        // Set up the http target
        CString strFmt;
        strFmt.Format (IDS_TWITTER_STATUS, szUserName);
        curl_easy_setopt (curl, CURLOPT_URL, strFmt.GetString ());
 
        // Save the result into memory for now.
        CMemFile buffer;
        curl_easy_setopt (curl, CURLOPT_WRITEFUNCTION, _CurlWriteCB);
        curl_easy_setopt (curl, CURLOPT_WRITEDATA,     &buffer);
 
        // Attempt to grab the data from Twitter.
        CURLcode res = curl_easy_perform (curl);
        curl_easy_cleanup (curl);
 
        // Return the data.
        if (res == 0) {
            uiSize = (UINT)buffer.GetLength ();
            return buffer.Detach ();
        }
    }
 
    return NULL;
}

The above listing will download a users tweets and store them in a growable buffer (see CMemFile). But now we have to present this to Xerces in a way that it will understand. Thankfully we can supply an arbitrary InputSource to a DOMParser, including one that will wrap a piece of memory.

bool DoGetStatuses (char* szUserName)
{
    // Query Twitter
    UINT  uiSize;
    BYTE* pbData = GetStatusesFromTwitter (szUserName, uiSize)
    if (pbData == NULL)
        return false;
 
    // Move the memory into an object Xerces understands.
    MemBufInputSource* pDataSrc = new MemBufInputSource
        (pbData, uiSize, L"TwitterXML", true);
 
    // Parse the data
    XercesDOMParser parser;
    parser.setValidationScheme (XercesDOMParser::Val_Never);
    parser.setDoNamespaces (false);
    parser.setDoSchema (false);
    parser.setDoValidation (false);
    parser.parse (*(InputSource*)pDataSrc);
 
    // Get the root node
    DOMDocument* pDoc = parser.getDocument ();
 
    //...
 
    // Free memory.
    delete pDataSrc;
    return true;
}

I was especially lazy in that last listing actually, because I told Xerces to adopt my buffer which means it’ll free it for me when it’s finished with it. Curiously though, even a memory object needs a system id which is the purpose L”TwitterXML” serves. With a DOMDocument in memory it was trivial to add the statuses to a list box.

I was quite surprised at how complex a task I’d achieved given the effort I’d put in; hats off to both Xerces and libcurl. Now that I’d managed to list my tweets, naturally the next step is to try and submit one! So I made a new dialog for the occasion:

Clicking OK causes some magic to happen:

int CPostStatusDlg::DoStatusUpdate ()
{
    static const char* cszUrl =
        "http://api.twitter.com/1/statuses/update.xml";
 
    // Attempt to initialise cURL.
    CURL* curl = curl_easy_init ();
    if (curl == NULL)
        return 0;
 
    // Configure the authentication
    CString strFmt;
    strFmt.Format (_T("%s:%s"), m_strUserName, m_strPassword);
    curl_easy_setopt (curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
    curl_easy_setopt (curl, CURLOPT_USERPWD,  (char*)strFmt.GetString ());
 
    // Format the string entire in C form.
    char* szStatus = curl_easy_escape (curl, m_strStatus.GetString (),
        m_strStatus.GetLength ());
    char szPostBody[BUFSIZ];
    sprintf (szPostBody, "status=%s", szStatus);
    curl_free (szStatus);
 
    // Set up the HTTP connection and use the POST method
    curl_easy_setopt (curl, CURLOPT_POST,           1L);
    curl_easy_setopt (curl, CURLOPT_POSTFIELDSIZE,  strlen (szPostBody));
    curl_easy_setopt (curl, CURLOPT_POSTFIELDS,     szPostBody);
 
    // Finally, set the callback function and the URL.
    CMemFile buffer;
    curl_easy_setopt (curl, CURLOPT_WRITEFUNCTION, _CurlWriteCB);
    curl_easy_setopt (curl, CURLOPT_WRITEDATA,     &buffer);
    curl_easy_setopt (curl, CURLOPT_URL,           cszUrl);
 
    // Now we can execute at last!
    int nResponse = 0;
    CURLcode res = curl_easy_perform (curl);
    curl_easy_getinfo (curl, CURLINFO_RESPONSE_CODE, &nResponse);
    curl_easy_cleanup (curl);
 
    // Check for success
    return nResponse;
}

Boom! Tweet submitted!

In the above listing, the growable buffer and the callback are largely to just eat the output from libcurl because we don’t really care about it. CMemFile will free the memory it allocated when the function returns too, which saves hassle. I originally wrote all the code listings with Unicode in mind which is why they might appear to be a bit odd. libcurl is an ANSI C library so you may need to convert your strings for it to work. Thankfully Xerces includes some basic support because it uses Unicode internally.

Little victory indeed.

I drafted a post about my adventures with trying to compile some open source software, but it turned out far too long and it didn’t actually concentrate on the point I was trying to make. So instead I’ll jump right to it.

Yesterday I decided, randomly, to have a look at libcurl. It’s a surprisingly powerful library for C that supports a huge amount of internet protocols with only a few function calls. The code examples on the website made it look easy enough to use, so I thought I’d download it and see what I can do with it. Unfortunately compiling it myself turned out to be annoyingly difficult, so I just downloaded the binaries. Naturally these were all in MinGW form and I needed them them to be MSVC compatible.

After some googling I found that MinGW comes with a tool called pexports which dumps all the symbols of a DLL. So using that I wrote a Bash script that does this for all the DLLs in a directory:

for i in $( ls | grep ".*\.dll" | sed "s/.dll//" ); do
    pexports "$i.dll" > "$i.def"
    echo "lib /machine:i386 /def:$i.def" >> makelib.bat
done

This creates a .def file for each DLL which lists all the symbols that must be exported, and adds a line to makelib.bat which will create the MSVC import library. The two pronged approach is unfortunately necessary here because the Microsoft LIB tool won’t run through the Cygwin prompt.

The first line might seem complicated, but it’s not too bad. ls lists all the files in the current directory, grep then singles out all the .dll files, and finally sed removes the trailing .dll. In the loop itself $i will therefore always be of the form “<mydll>” instead of “<mydll>.dll“.

I’m no expert with Bash scripting so there’s probably an easier way to do it.

I can’t remember what I was doing now, but for some reason I needed to copy the full path of some file into some program to do some… thing. I was finding it increasingly annoying that I had to copy the path from the Explorer window, then either hand-transcribe (complete with mistakes) or rename, select all, copy the file name.

“Why can’t I just right click and select Copy Filename or something?” I said to myself, “I wonder…”

So I searched CodeProject for some information on how to write my own damn Shell Extension (with Blackjack and Hookers) and stumbled across this fantastic article by Michael Dunn. In the space of an hour I had managed to learn enough to make half of my extension: I added a context menu item to Explorer!

Now over to MSDN to learn about the Clipboard and history was made. After around an hour and a half I’d gone from knowing nothing about writing Shell Extensions or using the Clipboard to having a working Shell Extension that used the Clipboard. I love the internet.

This is one of the smallest pieces of code I’ve ever written and, ironically, one of the few projects I consider myself to have finished. For more information, including how to download it, go to the CopyExt page.

I’ve bitten off a fair amount of stuff to do over the summer though. I’m currently nursing my interest in web development by helping to make the website for the unofficial ECS TF2 server. It’s quite nostalgic really, I haven’t done any proper web based stuff since school. I’m quite surprised how much things have changed, for the better thankfully.

A mock-up of my design idea can be found here for those that care. Feel free to vote for a banner. The palette is based entirely on one I found at Fabry Design but I got fed up with constantly using the eyedropper tool to find the colour values so I made a HTML version. That and I wanted to show off.

I’ve recently discovered jQuery which allows one to “write less, do more” and it really does deliver. I’ve used it on my design idea to make the palette and that vote box. It handles a surprisingly large about of stuff too, ranging from simple CSS manipulation through animations and up to AJAX. With a good enough backend it would be very easy to write an internet application using jQuery instead of Flash or Java.

AJAX is something I’ve wanted to try for a while now because it allows you to update a web page without needing to refresh it. If anyone has ever used WordPress then you’re probably familiar with the auto-save draft feature when writing a post. If it didn’t use AJAX your page history would fill up with bullshit.

So what exactly is AJAX? Like most acronyms in technology it has a scary name: Asynchronous JavaScript And XML. It’s asynchronous because you can handle an AJAX transaction at any point you like and it won’t refresh the page. You write the front-end code in JavaScript and typically communicate using XML. So, as usual, its bark is worse than its bite.

Using my ECS design prototype as an example: I use AJAX to submit votes and build that form but without refreshing the page. Every five seconds a new request is issued to the server to update the form values. This happens without any user interaction, or asynchronously. There are plenty of other uses for this kind of thing, the Facebook instant messenger probably uses it. I never log Facebook so I don’t know.

XML (eXtensible Markup Language) however is pretty bulky. I won’t go into it here because I can’t be bothered, but trust me that it is. Thankfully AJAX is not bound to XML despite it being present in its name, instead we can choose to use JavaScript Object Notation (JSON) which is much smaller. It’s supported natively by JavaScript which means it’s fast and easy to turn handle.

I intend to use what I’ve talked about here to gradually remake this site. My Experimental page will hopefully start filling up with stuff until I have enough to assemble a new website.

Word.

An unfortunate side effect of this atrocity is that you must now register to post comments. Every cloud has a silver lining however: now you shameless, friendless people with an increased sense of self-worth can’t post plugs to your boring blog that would bring a sloth to tears.

End rant, moving on…

Finally, D4 (our big project of the year) is done and dusted! It was a pretty interesting couple of weeks, despite being filled with stress, aggrovation and late nights. We had to design a fire detection system that could accurately measure temperature, detect smoke and the presence of people from up to four sensors and display that information on a web interface.

The basic design consisted of up to four Sensor nodes that contained a thermistor (measures temperature), smoke sensor from a butchered smoke alarm, and a PIR (Passive InfraRed) detector, and the Central Node which then connected to a PC. Each node had to have a memory to store data in the event of a power or communications failure, and the PC was used to control and monitor the whole system. A database (well, CSV file…) was used to store data from all sensors on the PC and that same file was used by the web interface to display tables and graphs of what was going down.

All in a week…

Regardless of the crazy timescale and annoying setbacks, we managed to get a fairly good prototype up and running. It was my job to build the Central Node and the software on the PC (named Monitor, see how imaginative I am?) which taught me a lot.

I spent the first day trying to get my unconventional approach to a USB interface working correctly. The documentation is a bit sparse, but fairly good nonetheless. I managed to use Command Endpoint 0 to control the state of some LED’s with relative ease, but I had a small amount of difficulty using Interrupt-In Endpoint 1 to send data to the PC when I flicked some switches. I got it down eventually though, so I was happy with my first day of work.

Shame this didn’t continue… I have no idea what happened to Tuesday and Wednesday. I just remember taking a particularly long walk home on Wednesday evening thinking “What have I actually achieved?” I was walking with my housemate, and the weather was good, so it was a good excuse to escape from stress and work.

Thursday was eventful as hell! We arrived in the morning with very small parts of the system working but nothing major going on. Matt and I joined forces. In a single day we had a fully working datapath that could accurately measure temperature to within half a degree Celsius! It also supported the necessary stuff to allow Smoke and PIR warnings to be sent to the computer. I had half written a piece of software to read the data from the Central Node and save it to a file, but it wasn’t good enough for the specification.

Thursday night was a long bastard. I completely rewrote my broken software into a multithreaded slice of moist cake that handled more than it needed to, just in case. I made it detect the Central Node being unplugged from the computer so it could ASK for lost data! I was, and still am, pretty pleased with it.

Friday, deadline day. First thing in the morning I tested my newly written software, and much to my surprised it worked fairly well. Needed a few changes and tweaks but it performed nicely. By this time we had most of the components we needed, just a case of making the firmware a bit more capable, but we needed to give all the nodes some memory. Our chosen device was a Philips I²C EEPROM because we were rocking the serial technologies in this lab. We used

• USB as the interface between the Central Node and PC,
• UART to link the Central Node and Sensor Nodes,
• SPI for the Sensor Nodes to control an ADC, and
• I²C to give each node some memory.

Which turned out to be awesome, grand total of seven pins used on a 28 pin AVR microcontroller!

Matt wrote a memory controller which we spent the morning trying to debug but, unfortunately, couldn’t get it to work despite using recommended code from Atmel themselves! By lunch it still wasn’t working so we decided to cut our losses and get a system together.

We had a few hiccups when getting the Sensor to send two bytes of data, but that was my sloppy programming. I tried to compensate for a single error in two places resulting in the same byte being recieved twice. Never mind eh? After that crease was ironed out successfully we had a working data path from the Sensor (didn’t have time to make another to test our awesome bespoke communications protocol) to the PC software. The web guys brought their software over and the other guys bought the smoke and PIR sensors over and both interfaced nicely. A few small errors, but nothing major.

And that was it! We demonstrated our end-to-end system successfully… until Brad melted our thermistor! Only a minor setback however, a quick dash to stores by John sorted that out. Now the system was back into it’s (unfortunately memory lacking) glory and we got some nice ticks in boxes. Job done. How anticlimactic.

Well that was longer than I expected. Easter is upon me now, so I intend to get into the swing of updating this often. Bet that won’t last long.

Everything feels the need to hold your hand through every single activity you might want to do and most don’t even offer a way of disabling these pointless features. I probably wouldn’t complain if I could see even a wafer-thin shred of usefulness, but I genuinely can’t. Typical example of this being that immensely annoying protection method of everyone’s favourite IM client: MSN Messenger. Whoever the hell decided that it should decide to block access to some files with certain extensions to “protect your security” should be shot in the leg and thrown to the rapists.

“Protect your security,” what a ridiculous statement.

So someone sends me an MP3 file that they’ve recorded themselves and asks for my opinion, but of course I can’t just fucking click “Open blah.mp3″ because that would cause MSN to block access to the file to protect my security. So I need to open My Recieved Files every single time someone sends me any file that isn’t a picture. Funny how it can be circumvented so easily but can’t be turned off, even if you have a certified Virus Scanner installed. Seriously, how is this a good feature? All it adds to the mix is a massive heaped dose of for fucks sake. Windows has that insultingly hand-holding Security Centre but at least you can turn it off.

Some programs even think they have the right to control your computer for you. That’s novel isn’t it? Far too often, I’ve been trying to log into something using one of those supposed-to-be-secret password things only to have some program steal the top-level focus and demand it be updated. Why the fucking hell do programmers make this feature so aggressive? Most of the time the programs that demand attention are programs I don’t even use often. DivX is a prime example of that. Only time I ever use DivX is when I feel the need to watch a game or film trailer in high-definition and considering both get churned out at mass-production studios that is a very rare occurrence.

So I’m trying to check my email and DivX grabs the foreground, eats half of my no longer secret password, and demands I update it due to security issues… Ironic really isn’t it?

When I installed it I was not prompted with the question “Do you want DivX to check for updates?” Nor was I prompted with “Do you want DivX to start with Windows?” Far too many programs just whack themselves in the list of shit that starts with Windows just to bog your system down without asking. Only programs I want to start with Windows are Winamp, Messenger, Steam and AVG. Yet I look in my StartUp folder (the archaic method) to find three programs, none of which I’ve approved to launch on boot; and sixteen in the registry. Three of those are NVidia related and two reference exactly the same program as one shortcut in StartUp!

Where’s the manners gone in software development? Back in simpler times when Windows was young, temperamental, unstable and grey, applications were so much more friendly. They’d never grab your attention unless it was actually justified, and installers would ask you about everything and wouldn’t force you to install on the C drive to increase performance by 0.01%. Some applications would even sense if you’re in full screen mode and leave you the fuck alone if you were, but not these days. Any program will interrupt anything you’re doing for any reason whatsoever.

In the middle of writing a document? Sure, it’ll interrupt, eat the rest of your sentence and possibly even crash Word for you.

In the middle of a heated battle in some awesome game? Hells yes it’ll interrupt, probably get you killed and crash the game for you if you’re lucky.

In the middle of a download that you thought you could leave going while you nip downstairs to get a cup of tea? Of course! Windows update will run, install updates and restart your computer without the foresight of asking your permission!

Oh the wonders of the digital age!

SPICE is a circuit simulator and it’s fairly good. Like all good systems it makes mistakes every now and then but we can overlook that in most cases. So I tried to test my understanding of how a Zener diode works by designing a really simple shunt regulator. It’s not much use in the real world but it can supply 100mA at 5V (well, 4.7V because Zener ratings are strange) to some load.

Theory says that the Zener will stop acting like a regulator when no current flows through it, and this happens when the voltage across it fails to exceed its breakdown voltage. As shown in the above circuit diagram, R1 and R2 form a potential divider, hence the voltage across R2 is V1*R2/(R1+R2). The breakdown voltage is roughly 5V as mentioned earlier, therefore the device will stop conducting when R2/(R1+R2) <= 5/12.

Personally, I think this graph is pretty awesome, even if it is completely unlabelled. Up the left hand side is the output voltage (i.e. the voltage across R2) and along the bottom is the resistance of R2. The curve bends quite sharply at about 50 Ohms which works out to be exactly what we predicted earlier: 50/(70+50)=5/12.

I’m impressed with LTspice. This took like 5 minutes to do. Most of the tools we use are fairly complicated, but this is pretty easy. It draws graphs well and it’s easy to get the plots you actually want rather than plotting everything against time. The only downside is that it doesn’t support parametric sweeps in the GUI so you have to type it in manually, but overall it seems pretty cool.