Starbeamrainbowlabs

Stardust
Blog


Archive

Mailing List Articles Atom Feed Comments Atom Feed Twitter

Tag Cloud

3d account algorithms announcement archives arduino artificial intelligence assembly async audio bash batch blog bookmarklet booting c sharp c++ challenge chrome os code codepen coding conundrums coding conundrums evolved command line compiling css dailyprogrammer debugging demystification distributed computing downtime embedded systems encryption es6 features event experiment external first impressions future game github github gist graphics hardware meetup holiday html html5 html5 canvas interfaces internet io.js jabber javascript js bin labs learning library linux low level lua maintenance network networking node.js operating systems performance photos php pixelbot portable privacy programming problems project projects prolog protocol pseudo 3d python reddit reference release releases resource review rust secrets security series list server servers software sorting source code control svg technical terminal textures three thing game three.js tool tutorial twitter ubuntu university update upgrade version control visual web website windows windows 10 xmpp

Demystifying UDP

Yesterday I was taking a look at [UDP Multicast], and attempting to try it out in C#. Unfortunately, I got a little bit confused as to how it worked, and ended up sending a couple of hours wondering what I did wrong. I'm writing this post to hopefully save you the trouble of fiddling around trying to get it to work yourself.

UDP stands for User Datagram Protocol (or Unreliable Datagram Protocol). It offers no guarantee that message sent will be received at the other end, but is usually faster than its counterpart, TCP. Each UDP message has a source and a destination address, a source port, and a destination port.

When you send a message to a multicast address (like the 239.0.0.0/8 range or the FF00::/8 range for ipv6, but that's a little bit more complicated), your router will send a copy of the message to all the other interested hosts on your network, leaving out hosts that have not registered their interest. Note here that an exact copy of the original message is sent to all interested parties. The original source and destination addresses are NOT changed by your router.

With that in mind, we can start to write some code.

IPAddress multicastGroup = IPAddress.Parse("239.1.2.3");
int port = 43;
IPEndPoint channel = new IPEndPoint(multicastGroup, port);
UdpClient client = new UdpClient(43);
client.JoinMulticastGroup(multicastGroup);

In the above, I set up a few variables or things like the multicast address that we are going to join, the port number, and so on. I pass the port number to the new UdpClient I create, letting it know that we are interested in messages sent to that port. I also create a variable called channel, which we will be using later.

Next up, we need to figure out a way to send a message. Unfortunately, the UdpClient class only supports sends arrays of bytes, so we will be have to convert anything we want to send to and from a byte array. Thankfully though this isn't too tough:

string data = "1 2, 1 2, Testing!";
byte[] payload = Encoding.UTF8.GetBytes(data);
string message = Encoding.UTF8.GetString(payload);

The above converts a simple string to and from a byte[] array. If you're interested, you can also serialise and deserialise C♯ objects to and from a byte[] array by using Binary Serialisation. Anyway, we can now write a method to send a message across the network. Here's what I came up with:

private static void Send(string data)
{
    Console.WriteLine("Sending '{0}' to {1}.", data, destination);
    byte[] payload = Encoding.UTF8.GetBytes(data);
    Send(payload);
}
private static void Send(byte[] payload)
{
    client.Send(payload, payload.Length, channel);
}

Here I've defined a method to send stuff across the network for me. I've added an overload, too, which automatically converts string into byte[] arrays for me.

Putting the above together will result in a multicast message being sent across the network. This won't do us much good though unless we can also receive messages from the network too. Let's fix that:

public static async Task Listen()
{
    while(true)
    {
        UdpReceiveResult result = await client.ReceiveAsync();
        string message = Encoding.UTF8.GetString(result.Buffer);
        Console.WriteLine("{0}: {1}", result.RemoteEndPoint, message);
    }
}

You might not have seen (or heard of) asynchronous C# before, but basically it's a ways of doing another thing whilst you are waiting for one thing to complete. Dot net perls have a good tutorial on the subject if you want to read up on it.

For now though, here's how you call an asynchronous method from a synchronous one (like the Main() method since that once can't be async apparently):

Task.Run(() => Listen).Wait();

If you run the above in one program while sending a message in another, you should see something appear in the console of the listener. If not, your computer may not be configured to receive multicast messages that were sent from itself. In this case try running the listener on a different machine to the sender. In theory you should be able to run the listener on as many hosts on your local network as you want and they should all receive the same message.

Reading HTTP 1.1 requests from a real web server in C#

I've received rather a lot of questions recently asking the same question, so I thought that I 'd write a blog post on it. Here's the question:

Why does my network client fail to connect when it is using HTTP/1.1?

I encountered this same problem, and after half an hour of debugging I found the problem: It wasn't failing to connect at all, rather it was failing to read the response from the server. Consider the following program:

using System;
using System.IO;
using System.Net.Sockets;

class Program
{
    static void Main(string[] args)
    {
        TcpClient client = new TcpClient("host.name", 80);
        client.SendTimeout = 3000;
        client.ReceiveTimeout = 3000;
        StreamWriter writer = new StreamWriter(client.GetStream());
        StreamReader reader = new StreamReader(client.GetStream());
        writer.WriteLine("GET /path HTTP/1.1");
        writer.WriteLine("Host: server.name");
        writer.WriteLine();
        writer.Flush();

        string response = reader.ReadToEnd();

        Console.WriteLine("Got Response: '{0}'", response);
    }
}

If you change the hostname and request path, and then compile and run it, you (might) get the following error:

An unhandled exception of type 'System.IO.IOException' occurred in System.dll

Additional information: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time or established connection failed because connected host has failed to respond.

Strange. I'm sure that we sent the request. Let's try reading the response line by line:

string response = string.Empty;
do
{
    string nextLine = reader.ReadLine();
    response += nextLine;
    Console.WriteLine("> {0}", nextLine);
} while (reader.Peek() != -1);

Here's some example output from my server:

> HTTP/1.1 200 OK
> Server: nginx/1.9.10
> Date: Tue, 09 Feb 2016 15:48:31 GMT
> Content-Type: text/html
> Transfer-Encoding: chunked
> Connection: keep-alive
> Vary: Accept-Encoding
> strict-transport-security: max-age=31536000;
>
> 2ef
> <html>
> <head><title>Index of /libraries/</title></head>
> <body bgcolor="white">
> <h1>Index of /libraries/</h1><hr><pre><a href="../">../</a>
> <a href="prism-grammars/">prism-grammars/</a>
   09-Feb-2016 13:56                   -
> <a href="blazy.js">blazy.js</a>                                           09-F
eb-2016 13:38                9750
> <a href="prism.css">prism.css</a>                                          09-
Feb-2016 13:58               11937
> <a href="prism.js">prism.js</a>                                           09-F
eb-2016 13:58               35218
> <a href="smoothscroll.js">smoothscroll.js</a>
   20-Apr-2015 17:01                3240
> </pre><hr></body>
> </html>
>
> 0
>

...but we still get the same error. Why? The reason is that the web server is keeping the connection open, just in case we want to send another request. While this would usually be helpful (say in the case of a web browser - it will probably want to download some images or something after receiving the initial response), it's rather a nuisance for us, since we don't want to send another request and it's rather awkward to detect the end of the response without detecting the end of the stream (that's what the while (reader.Peek() != -1); is for in the example above).

Thankfully, there are a few solutions to this. Firstly, the web server will sometimes (but not always - take the example response above for starters) send a content-length header. This header will tell you how many bytes follow after the double newline (\r\n\r\n) that separate the response headers from the response body. We could use this to detect the end of the message. This is the reccommended way , according to RFC2616.

Another way to cheat here is to send the connection: close header. This instructs the web server to close the connection after sending the message (Note that this will break some of the tests in the ACW, so don't use this method!). Then we can use reader.ReadToEnd() as normal.

A further cheat would be to detect the expected end of the message that we are looking for. For HTML this will practically always be </html>. We can close the connection after we receive this line (although this doesn't work when you're not receiving HTML). This is seriously not a good idea. The HTML could be malformed, and not contain </html>.

Jabber & XMPP: A Lost Protocol

Welcome to a special tutorial post here at starbeamrainbowlabs.com. In this post, we will be exploring an instant messaging protocol known as XMPP.

The XMPP logo

Today, you will probably use something like Skype, Gmail, or possibly FaceTime to stay in touch with your friends and family. If you were to rewind to roughly the year 2000, however, you would find that none of the above existed yet. Instead, there was something called XMPP. Originally called Jabber, XMPP is an open decentralised communications protocol (that Gmail's instant messaging service uses behind the scenes!) that allows you to stay in touch with people over the internet.

Identifying Users

There are several programs and apps that have XMPP support built in, but first let's take a look how it works. As I mentioned above, XMPP is decentralised. This means that there is no central point at which you can get an account - in fact you can create your very XMPP server right now! I will go into the details of that in a future post. Having multiple servers also raises the question of identification. How do you identify all these XMPP users at hundreds, possibly thousands of server across the globe?

Several account at 2 different servers

Thankfully, the answer is really quite simple: We use something called a Jabber ID (JID), which looks rather like an email address, for example: bob@bobsrockets.com. Just like an email address, the user name comes before the @ sign, and the server name comes after the @ sign.

Connecting People

Now that we know how you identify an XMPP user, we can look at how users connect and talk to each other, even if they have accounts at different servers. Connecting users is accomplished by 2 types of connections: client to server (c2s) and server to server (s2s) connections, which are usually carried out on ports 5222 and 5269 respectively. The client to server connections connect a user to their server that they registered with originally, and the server to server connections connect the user's server to the server that hosts the account to the other user that they want to talk to. In this way an XMPP user may start a conversation with any other XMPP user at any other server!

A visualisation of the example below

Here's an example. Bob is the owner of a company called Bob's Rockets and has the XMPP account bob@bobsrockets.com. He wants to talk to Bill, who owns the prestigious company Bill's Boosters who has the JID bill@billsboosters.com. Bob will log into his XMPP account at bobsrockets.com over port 5222 (unless he is behind a firewall, but we will cover that later). Bill will log into his account at billsboosters.com over the same port. When Bob starts a chat with Bill, the server at bobsrockets.com will automagically establish a new server to server connection with billsboosters.com in order to exchange messages.

Note: When starting a conversation with another user that you haven't talked to before, XMPP requires that both parties give permission to talk to one another. Depending on your client, you may see a box or notification appear somewhere, which you have to accept.

Get your own!

Now that we have taken a look at how it works, you probably want your own account. Getting one is simple: Just go to a site like jabber.org and sign up. If you stick around for the second post in this series though I will be showing you how to set up your very own XMPP server (with encryption).

As for a program or app you can use on your computer and / or your phone, I recommend Pidgin for computers and Xabber for Android phones.

Next time, I will be showing you how to set up your own XMPP server using Prosody. I will also be showing you a few of the add-ons you can plug in to add support for things like multi-user chatrooms (optionally with passwords), file transfer proxies, firewall-busting BOSH proxies, and more!

Art by Mythdael