Starbeamrainbowlabs

Stardust
Blog


Archive

Mailing List Articles Atom Feed Comments Atom Feed Twitter Reddit Facebook

Tag Cloud

3d account algorithms announcement architecture archives arduino artificial intelligence artix assembly async audio bash batch blog bookmarklet booting c sharp c++ challenge chrome os code codepen coding conundrums coding conundrums evolved command line compilers compiling css dailyprogrammer debugging demystification distributed computing downtime electronics email embedded systems encryption es6 features event experiment external first impressions future game github github gist graphics hardware hardware meetup holiday html html5 html5 canvas infrastructure interfaces internet io.js jabber javascript js bin labs learning library linux low level lua maintenance manjaro network networking node.js operating systems performance photos php pixelbot portable privacy programming problems project projects prolog protocol protocols pseudo 3d python reddit reference release releases resource review rust secrets security series list server software sorting source code control statistics svg technical terminal textures three thing game three.js tool tutorial tutorials twitter ubuntu university update updates upgrade version control virtual reality virtualisation visual web website windows windows 10 xmpp xslt

Flexible Bison: Compiler Theory

One of the modules I've picked to do in my first semester of my third year at university is Lanuguages and their Compilers. Naturally, this entails building a compiler to compile a program that's written in a source language (spec provided, thankfully! :D) into plain old ANSI C.

The tools we're going to be using for this and the steps involved in actually compiling something into another language are somewhat complicated, and I'm having a bit of difficulty getting my head around the different steps a compiler goes through and how these steps relate to the tools we're going to be using. This blog post is my attempt to make sense of what I've learnt so far.

Firstly, let me introduce the tools I'll be using: GNU flex and GNU bison. Apparently they have a much shallower learning curve than other tools out there. At first, this doesn't appear to be the case - but the more I think about it the more I realise that this is true.

Flex, as far as I can tell, is a regular-expression based scanning tokeniser. In other words, it breaks down an input string into a series of tokens. It has a method that, when called, finds and returns the next token from the source string.

Bison uses tokenised output from flex to construct a parse tree. This parse tree is then optimised with redundant nodes removed, loops optimised, and other such tweaks. Finally, this optimised tree is then used to generate the output code.

With the cast introduced, I can get to the stages of a compiler:

  1. Lexical Analysis - Tokenisation
  2. Syntactical Analysis - Conversion of the token stream into a parse tree
  3. Semantic Analysis - Correction of the tree - e.g. automatic type conversion
  4. Intermediate Code Generation - Sometimes the compiler outputs sets of 3 values in a list of tuples. This was needed in older computers that couldn't hold all the steps of a compiler in memory at once! In my case, I'll be outputting the parse tree generated in step 3 I guess - but not to disk, as today we can have all the passes of the compiler in memory at the same time :D
  5. Optimisation - Redundant parts of the parse tree are removed etc. - loops are focused in particular
  6. Code Generation - The output code in the target language is generated here - whether that be in C (very common), Assembly, or another language.

This seems somewhat familiar. The Lexical Analysis phase seems to be rather similar to what flex is designed for, and the Semantic Analysis stage appears to what bison does. As for the other stages, I'm not really sure. I'm guessing that it'll become clear later as we build this compiler in stages - but I'm suspecting that we'll be writing them in plain C - unless I've missed something about bison :P

If you've made it this far, thanks for reading! If this feels somewhat disorganised - then it probably is - after all, this is mainly to get it all straight in my own head :P

If you've got any questions, please ask away in the comments below :-)

University: Begin!

A nice flowerbed at university last year :D Since I start my third year at university on Monday, I thought I'd make a quick post here about what you can expect to see on here in the future. If you're starting another (or your first!) year of university this year, I wish you good luck!

In the first semester, I've chosen a pair of modules about languages and their compilers and virtual reality. I thought the former sounded quite cool - I'm hoping that I'll end up understanding what goes on under the hood in the compilers that power the languages we use today. I haven't had much exposure to the latter - so I thought that it would be a good introduction to the subject to 'broaden my horizons', so to speak - that is to say I'm curious to investigate an area that I haven't touched before.

For the second semester, I've chosen a mobile development module and an advanced AI module. Personally I'm most excited about these two - The Prolog that I did (and posted about!) before was actually really rather fun and made sense in a strange sort of way, so I thought I'd try my hand at the next level. Mobile development is another area that I've been interested in experimenting with - I've been pondering writing an Android app for Pepperminty Wiki, my lightweight wiki engine that powers a personal project of mine.

With this in mind, you can expect to see a bunch of blog posts relating to these areas that I'll be exploring :D

Writing code when you don't have the time

As you've probably noticed, posts around here have slowed down recently. There's a reason: I've been very busy doing a year in industry. Currently, my goal is to release one post a week. While my time has been rather fragmented and at times extremely limited, I've still been able to sit down for a little while here and there to write some blog posts and some code (If I can actually pull it off, I've got a seriously cool project I'm going to post about on here in the near-ish future!).

Due in part to the fact that I really don't want to exclusively write code at my industrial placement, I've been trying my hardest to keep programming and playing around with things in my free time. It's not as easy as you might think. Sometimes, the setup and teardown time eats all the time I allocated away so can't actually get anything done.

If this sounds a little bit like your time at the moment, fear not! I have developed a technique or two I wanted to share on here, just in case someone else finds it useful :-)

Planning what it is that you want to do is really important. You probably know this already, but it is especially so if you don't have a ton of time to throw at a project, because otherwise you can easily spend longer figuring out what you need to do next than actually doing it. I try to break my projects down into small, manageable bite-sized chunks that I can tackle one at a time. Only have 1/2 an hour at a time? Break it down into portions that will take you about 1/2 an hour complete. It might take a while, but breaking your project down can help it go a little bit faster.

Even with breaking my project down, I often find myself forgetting where I got to last time. To tackle this, I've discovered that leaving a comment in the file I was last editing explaining in a sentence or two what I need to do next helps me figure it out faster. It's also really useful that my editor (whichever one I'm using at the time) is configured to remember the files I had open last - letting me quickly pick up where I left off. Monodevelop, Visual Studio, and Atom do this automatically - if your editor doesn't, there's bound to be a setting or an extension that does it for you.

By planning what I need to do next, and leaving myself short comments explaining what I was about to do next, I can increase the amount of time I spend actually writing code instead of fumbling around working out what I wanted to do next. It's certainly not an ideal way to program, but with practice you can get quite proficient at it....

Found this helpful? Got any tips yourself? Comment down below!

I've got some business cards!

My new business cards!

I've been to several events of various natures now (like the Hardware Meetup), and at each one I've found that a 'business card' or two would be really handy to give to people so that they can remember the address of this website.

After fiddling with the design over about a week I (and Mythdael!) came up with the design you see above. Personally, I'm really pleased with them, so I decided to post here to show them off :-)

I'm finding that it's a rather good idea to promote and build your brand at these kind of events by showing people the cool things that you've created and learnt, and business cards seem to be just the thing that helps you do it.

A close up of the front and back of my new business cards.

How to run a successful blog: My experiences so far

About 2 and a half years (and 207 posts!) ago today, I took the advice of Rob Miles and started this blog. I've learned a lot since I started from some wonderful people, and the quality of my posts on here has improved immensely. I've been meaning to write a post on what I've learnt about running this blog, but haven't gotten around to it until now :-)

The first, and most important, thing you can do is to post often. By posting often you keep people coming back to read more. If you leave it too long between posts, people may forget about your blog and may not come back.

Of course, while having regular content is good, having a posting schedule is considered by some to be even better. While I'm not particularly good at this yet, if you do manage to achieve it you can ensure that people know when they can come back and expect another post.

It's also important that you post about something you find interesting. Doing so improves the quality of the post, and it makes for a much more interesting read. If you're passionate about what you're writing about, people will generally come back for more.

Next, blogging is something that you have to persevere with. You certainly won't acquire readers overnight. It's worth bearing in mind that most of your readers will be silent - only a very small proportion of readers will actually leave a comment, no matter how easy you make the process (I hardly ever get comments).

Also, don't set out with the goal of getting $x$ number of regular readers, or earning $y$ amount of money. They are bad motivators to have, and you will surely be disappointed. Rather, you should set out to just write about something you love, without expecting anyone to read what you've written. Then you'll be pleasantly surprised :D

That's all I can think of right now. There are surely other things that I haven't mentioned. If you have any tips you've learnt from running a blog of your own, post about them in the comments :D

My robot works!

The Hull Pixelbot 2.0!

I went to the hardware meetup again today. For the last 2 times I've either managed to forget or I've been on holiday (probably both...), but I remembered this time around and I decided to go. I'm glad I did, because, with Rob's help I fixed my robot! It is now trundling around on the floor quite happily :D

The problem was that I assumed that the digital pin numbers on the wemos D1 R2 were the same as the GPIO pin numbers, which isn't the case apparently. Here's a table:

Digital GPIO
D0 GPIO 16
D1 GPIO 5
D2 GPIO 4
D3 GPIO 0
D4 GPIO 2
D5 GPIO 14
D6 GPIO 12
D7 GPIO 13
D8 GPIO 15

(Table from wemos.cc)

Very odd numbering. It's rather frustrating actually! Anyway, there seemed to be quite a lot more people there this time as compared to last time - it looks like the idea is taking off, which I'm very glad about :-)

If you're in the Hull area and interested in hardware, you should seriously consider coming along. There's a page on meetup.com that contains the details of the next meetup, but the general rule of thumb is that it happens at C4DI every first and third Thursday of every month at 6pm - 8pm.

Before I forget, I'm going to end this post off by posting the modified version of Rob's hull pixelbot dance code that works on the Wemos:


(Git Repo, Raw)

I built a robot!

The robot I built! The day before yesterday we had another hardware meetup at C4DI, and I built a robot! Now I just have to figure out how to program it...

It's one of Rob Miles' Hull Pixel Bots, with a Wemos D1 R2 on the top and two stepper motors and their driver boards mounted in the middle, and a battery box on the bottom.

The wheels are a little wonky, but we'll sort that out next time :) For now, I'm going to have some fun making it run around on the floor :D

The Hull Pixelbot Meetup

Rob's Hull Pixelbot (Above: Rob's WiFi-enabled Pixelbot.)

Today Rob Miles was kind enough to give me a lift to the monthly hardware (or hull pixel bot) meetup. It was different to what I'm used to, but it was rather fun actually!

Rob Miles has built a kit that gives you the parts to build your very own Arduino-powered robot that trundles around on the floor. He's also managed to add a WiFi chip to it too - so you can (provided you write the code) connect to your pixel bot and control it remotely!

You can build your own by going to hullpixelbot.com.

I'll certainly be playing around with it and attending the next meetup (meetups are on the first Thursday of every month at 6:00pm at C4DI).

Demystifying UDP

Yesterday I was taking a look at [UDP Multicast], and attempting to try it out in C#. Unfortunately, I got a little bit confused as to how it worked, and ended up sending a couple of hours wondering what I did wrong. I'm writing this post to hopefully save you the trouble of fiddling around trying to get it to work yourself.

UDP stands for User Datagram Protocol (or Unreliable Datagram Protocol). It offers no guarantee that message sent will be received at the other end, but is usually faster than its counterpart, TCP. Each UDP message has a source and a destination address, a source port, and a destination port.

When you send a message to a multicast address (like the 239.0.0.0/8 range or the FF00::/8 range for ipv6, but that's a little bit more complicated), your router will send a copy of the message to all the other interested hosts on your network, leaving out hosts that have not registered their interest. Note here that an exact copy of the original message is sent to all interested parties. The original source and destination addresses are NOT changed by your router.

With that in mind, we can start to write some code.

IPAddress multicastGroup = IPAddress.Parse("239.1.2.3");
int port = 43;
IPEndPoint channel = new IPEndPoint(multicastGroup, port);
UdpClient client = new UdpClient(43);
client.JoinMulticastGroup(multicastGroup);

In the above, I set up a few variables or things like the multicast address that we are going to join, the port number, and so on. I pass the port number to the new UdpClient I create, letting it know that we are interested in messages sent to that port. I also create a variable called channel, which we will be using later.

Next up, we need to figure out a way to send a message. Unfortunately, the UdpClient class only supports sends arrays of bytes, so we will be have to convert anything we want to send to and from a byte array. Thankfully though this isn't too tough:

string data = "1 2, 1 2, Testing!";
byte[] payload = Encoding.UTF8.GetBytes(data);
string message = Encoding.UTF8.GetString(payload);

The above converts a simple string to and from a byte[] array. If you're interested, you can also serialise and deserialise C♯ objects to and from a byte[] array by using Binary Serialisation. Anyway, we can now write a method to send a message across the network. Here's what I came up with:

private static void Send(string data)
{
    Console.WriteLine("Sending '{0}' to {1}.", data, destination);
    byte[] payload = Encoding.UTF8.GetBytes(data);
    Send(payload);
}
private static void Send(byte[] payload)
{
    client.Send(payload, payload.Length, channel);
}

Here I've defined a method to send stuff across the network for me. I've added an overload, too, which automatically converts string into byte[] arrays for me.

Putting the above together will result in a multicast message being sent across the network. This won't do us much good though unless we can also receive messages from the network too. Let's fix that:

public static async Task Listen()
{
    while(true)
    {
        UdpReceiveResult result = await client.ReceiveAsync();
        string message = Encoding.UTF8.GetString(result.Buffer);
        Console.WriteLine("{0}: {1}", result.RemoteEndPoint, message);
    }
}

You might not have seen (or heard of) asynchronous C# before, but basically it's a ways of doing another thing whilst you are waiting for one thing to complete. Dot net perls have a good tutorial on the subject if you want to read up on it.

For now though, here's how you call an asynchronous method from a synchronous one (like the Main() method since that once can't be async apparently):

Task.Run(() => Listen).Wait();

If you run the above in one program while sending a message in another, you should see something appear in the console of the listener. If not, your computer may not be configured to receive multicast messages that were sent from itself. In this case try running the listener on a different machine to the sender. In theory you should be able to run the listener on as many hosts on your local network as you want and they should all receive the same message.

Tips on writing annoying / difficult code

If you've been programming for any length of time, you'll know that eventually you will have to implement a particularly complex algorithm or conquer some other especially difficult challenge. Maybe it's implementing specular lighting, or perhaps it's fixing a nasty annoying bug in your physics engine.

Since I seem to have been doing this with increasing frequency at the moment due to University coursework, I thought I'd share a few things with you that I personally find helpful.

Plan out what you intend to do. Create a flowchart. Fill a whiteboard with notes. Talk to someone. Even if it's a rubber duck. Planning out what you want to do ensures that you have a clear idea in your head as to what you want to accomplish and how to go about doing it. It's very easy to fall into the trap of doing random things and just hoping that it will work.

Break the task down into steps. This goes hand in hand with the above point. Break your task down into small, manageable steps that you can follow one at a time to achieve your goal. I often find if I don't do this the problem I'm trying to solve looks an awful lot bigger than it really is.

Tackle the steps one at a time, and test regularly. Skipping to step 3 when you've barely started step 1 might be tempting, but it will probably cause more problems that it solves and lead to confusion. If you're thinking of doing this, then you might not have your steps in the right order.

Similarly, doing steps 1-4 before testing what you've done also isn't a good idea, because you might have made a mistake in step 1 that affects the output of step 4, and then spend an hour debugging step 4's code only to realise it is fine and it's actually the code you wrote in the beginning that is causing the problem.

I hope you find these tips helpful. If you did, leave a comment below!

Art by Mythdael