Starbeamrainbowlabs

Stardust
Blog


Archive


Mailing List Articles Atom Feed Comments Atom Feed Twitter Reddit Facebook

Tag Cloud

3d account algorithms android announcement architecture archives arduino artificial intelligence artix assembly async audio bash batch blog bookmarklet booting c sharp c++ challenge chrome os code codepen coding conundrums coding conundrums evolved command line compilers compiling compression css dailyprogrammer debugging demystification distributed computing documentation downtime electronics email embedded systems encryption es6 features event experiment external first impressions future game github github gist gitlab graphics hardware hardware meetup holiday holidays html html5 html5 canvas infrastructure interfaces internet io.js jabber jam javascript js bin labs learning library linux lora low level lua maintenance manjaro network networking nibriboard node.js operating systems performance photos php pixelbot portable privacy problem solving programming problems projects prolog protocol protocols pseudo 3d python reddit redis reference release releases resource review rust searching secrets security series list server software sorting source code control statistics storage svg technical terminal textures three thing game three.js tool tutorial tutorials twitter ubuntu university update updates upgrade version control virtual reality virtualisation visual web website windows windows 10 xmpp xslt

File System Performance in PHP

While writing pepperminty wiki, I started seeing a rather nasty in crease in page load times. After looking into it, I drew the conclusion that it must have been the file system that caused the problem. At the time, I had multiple calls to PHP's glob function to find all the wiki pages in the current directory, and I was checking to see if the wiki page existed before reading it into memory.

The solution: A page index. To cut down on the number of reads from the file system, I created a json file that containedd inforamtion about every page on the wiki. This way, it only needs to check the existence of and read in a single file before it can start rendering any one page. If the page index doesn't exist, it is automatically rebuilt with the glob function to find all the wiki pages in the current directory.

In short: to increase the performance of your PHP application, try to reduce the number of reads (and writes!) to the file system to an absolute minimum.

I still need to update the code to allow users to delete pages via the GUI though, because at present you have to have access to the server files to delete a page and then remove it from the page index manually.

Tag Clouds

Yesterday I added a tag cloud to the website. You can find it at the top of the blog, or below the post on the article view. The tags are not yet linked to a list of posts that have the corresponding tags, as I need to some a litttle bit of refactoring of the blog's code in order to get that to work.

Pepperminty Wiki: A Wiki in a box

Recently I found a post on reddit by someone called am2064 about a 'one file wiki' called 'Minty Wiki' written in PHP. I took a look and whilie it was cool, I found it to have some bugs in it. I also found that it needed an extra PHP file to parse markdown to make it work properly. Still, I thought it was a cool idea so I decided to have a go myself.

694 lines of code later, I had something that worked and I thought that I might post about here on my blog. It is by no means finished, but it is in a somewhat usable (hopefully secure) state. I decided that markdown was the most logical choice for editing pages, so I modified Slimdown (by Johnny Broadway) to add internal link parsing and tweaked the bold/italics code to be mroe like Gmail's chat amongst other things. I first found Slimdown when looking for a lightweight markdown parser for comments on this blog.

I named my creation 'Pepperminty Wiki' (after the wiki that gave me the idea). It currently allows you to create and edit pages (although you need access to the server's files to delete pages currently), list all current pages, and view a printable version of a page. It even has a 'search' box that allows you to type in the name of the page you want to view. The search box has an HTML5 <datalist> to provide the autocomplete functionality.

To use it yourself, simply download index.php in the github repository below and put it in a folder on your server. Make sure that you have enabled write access to the folder though, or else you will start to see to rather strange error messages :)

To configure it, simply open the file you downloaded with your favourite text editor. You will find the settings (along with an explanation of each) at the top of the file. Make sure that you change the usernames and passwords!

You can find it on github here: Pepperminty Wiki

A (uneditable) version can be found here: Demo

Soon I will write up a technical post about my efforts to improve the performance of Pepperminty Wiki.

Github!

I now have a github account. I will be using it to house programming projects, most of which are / will be released on this website.

You can find it here: https://github.com/sbrl/

I currently have 1 repository, which holds something that will be released here soon.

HTML5 Canvas Clouds

This week I have some clouds for you, rendered via the HTML5 <canvas>. I wrote these in early 2013. I have not had a lot of time this week, but something cool is coming soon :)

The clouds themselves are stored in an array, and are composed of a random number of circles. This array is then iterated over 60 times a second and rendered using the HTML5 <canvas>. setInterval() is used to schedule the drawing of the frames, but I really should go back and upgrade that to requestAnimationFrame().

Link: HTML5 Canvas Clouds

Easy C♯ Menus

I have written some easy to use C♯ menus, and I thought that I would post about them here.

There are currently 2 different methods (and 2 extra helper methods). Both methods also centre the dialog box in the middle of the console, and also don't produce much mess that needs cleaning up afterwards.

Firstly, here are the two helper methods:

//from https://stackoverflow.com/questions/17590528/pad-left-pad-right-pad-center-string
///<summary>
///Cool function from stackoverflow that pads a string on both sides to make it a given legnth.
///</summary>
///<param name="source">The source string to pad.</param>
///<param name="length">The desired length.</param>
///<returns>The padded string.</returns>
static string PadBoth(string source, int length)
{
    int spaces = length - source.Length;
    int padLeft = spaces/2 + source.Length;
    return source.PadLeft(padLeft).PadRight(length);
}

//utility function that uses the above to pad a string to the current width of the console.
static string PadToWindowWidth(string str)
{
    return PadBoth(str, Console.WindowWidth - 1);
}

These need to be included in addition to either (or both!) of the methods described below.

A screenshot of the first menu type. The first one is a flexible multiple choice selection window. You can pass in an array of string sthat you want the user to choose from, and the method will deal with the rest, returning the index in the array of the item that the user chose.

This method comes with support for an optional prompt to display at the top of the dialog box. If you omit it, the prompt will not be displayed.

Source code:

///<summary>
///Displays a nice multiple choice menu that the user can intract with using the arrow keys.
///</summary>
///<param name="options">An array of strings that should be used as the possible options in the menu.</param>
///<returns>The index of the option the user chose.</returns>
static int DisplayMenu(string[] options, string prompt = "")
{
    int cursorstartx = Console.CursorLeft,
        cursorstarty = Console.CursorTop,

        menuWidth = (int)(Console.WindowWidth * 0.8),
        menuHeight = 2 + (options.Length * 2) + 1;

    if(prompt.Length > 0)
        menuHeight += 2;

    int currentIndex = 0;

    string menu = "";

    while(true)
    {
        Console.SetCursorPosition(cursorstartx, cursorstarty);

        menu = "";
        menu += new String('\n', ((Console.WindowHeight - 1) - menuHeight) / 2);
        menu += PadToWindowWidth("".PadLeft(menuWidth, '-')) + "\n";

        if(prompt.Length > 0)
        {
            menu += PadToWindowWidth("| " + PadBoth(prompt, menuWidth - 4) + " |") + "\n";
            menu += PadToWindowWidth("".PadLeft(menuWidth, '-')) + "\n";
        }

        menu += PadToWindowWidth("|" + "".PadLeft(menuWidth - 2) + "|") + "\n";

        for(int i = 0; i < options.Length; i++)
        {
            if(currentIndex == i)
            {
                menu += PadToWindowWidth("|" + PadBoth("> " + options[i] + " <", menuWidth - 2) + "|") + "\n";
            }
            else
            {
                menu += PadToWindowWidth("|" + PadBoth(options[i], menuWidth - 2) + "|") + "\n";
            }
            menu += PadToWindowWidth("|" + "".PadLeft(menuWidth - 2) + "|") + "\n";
        }
        menu += PadToWindowWidth("".PadLeft(menuWidth, '-'));
        menu += new String('\n', ((Console.WindowHeight - 1) - menuHeight) / 2);

        Console.WriteLine(menu);

        ConsoleKeyInfo nextkey = Console.ReadKey(true);

        switch(nextkey.Key.ToString())
        {
            case "UpArrow":
                currentIndex--;
                break;

            case "DownArrow":
                currentIndex++;
                break;

            case "Enter":
                return currentIndex;
        }

        if(currentIndex < 0)
            currentIndex = options.Length - 1;
        if(currentIndex > options.Length - 1)
            currentIndex = 0;
    }
}

Confirm Dialog

A screenshot of the second menu type.

In case you want to obtain an answer to a simple yes/no question, this second method allows you to ask the user to choose between 2 choices. Simply specify a prompt, and optionally the text to display in the place of the "Yes" / "No", and the method will return true or false, depending on which one the user selected.

Source code:

///<summary>
///Asks the user a simple yes/no question in the form of a console based dialog box.
///</summary>
///<param name="prompt">The question to ask the user.</param>
///<param name="trueText">The text to display in the place of "Yes"</param>
///<param name="falseText">The text to display in the place of "No"</param>
///<returns>True if the user selected "Yes", or false if the user selected "No".</returns>
static bool DisplayConfirm(string prompt, string trueText = "Yes", string falseText = "No")
{
    int cursorstartx = Console.CursorLeft,
        cursorstarty = Console.CursorTop,

        menuWidth = (int)(Console.WindowWidth * 0.8),
        menuHeight = 7,

        currentIndex = 1;

    string menu = "";

    while(true)
    {
        Console.SetCursorPosition(cursorstartx, cursorstarty);

        menu = "";
        menu += new String('\n', ((Console.WindowHeight - 1) - menuHeight) / 2); //vertical centring

        menu += PadToWindowWidth("".PadLeft(menuWidth, '-')) + "\n"; //dashes
        menu += PadToWindowWidth("|" + "".PadLeft(menuWidth - 2) + "|") + "\n"; //space

        menu += PadToWindowWidth("|" + PadBoth(prompt, menuWidth - 2) + "|") + "\n"; //prompt
        menu += PadToWindowWidth("|" + "".PadLeft(menuWidth - 2) + "|") + "\n"; //space
        menu += PadToWindowWidth("".PadLeft(menuWidth, '-')) + "\n"; //dashes

        if(currentIndex == 0)
            menu += PadToWindowWidth("|" + PadBoth(trueText, (menuWidth - 3) / 2) + "|" + PadBoth("> " + falseText + " <", (menuWidth - 3) / 2) + "|") + "\n";
        else
            menu += PadToWindowWidth("|" + PadBoth("> " + trueText + " <", (menuWidth - 3) / 2) + "|" + PadBoth(falseText, (menuWidth - 3) / 2) + "|") + "\n";

        menu += PadToWindowWidth("".PadLeft(menuWidth, '-')) + "\n"; //dashes

        menu += new String('\n', ((Console.WindowHeight - 1) - menuHeight) / 2); //vertical centring

        Console.WriteLine(menu);

        ConsoleKeyInfo nextkey = Console.ReadKey(true);

        switch(nextkey.Key.ToString())
        {
            case "LeftArrow":
            case "UpArrow":
                currentIndex--;
                break;

            case "RightArrow":
            case "DownArrow":
                currentIndex++;
                break;

            case "Enter":
                if(currentIndex == 0)
                    return true;
                else
                    return false;
        }

        if(currentIndex < 0)
            currentIndex = 1;
        if(currentIndex > 1)
            currentIndex = 0;
    }
}

I will probably write a few more of these in the future, and make these current ones better.

Demonstration binaries are available upon request, simply leave a comment below.

Friday Afternoon Downtime

Midafternoon today, starbeamrainbowlabs.com was down. The reason for this is unclear, but it seems that this site's hosting provider, OVH, was doing some maintenance. The site was down for 29 minutes.

Lessons Learned

Get your server to notify you during the shutdown / startup sequences

Using a service like Uptime Robot is great to monitor your website's uptime, but it is a much better idea to have your server send you an email when your server goes down. That way you will know whether the server got a chance to shut down correctly or not.

Backup often, and before / after major changes

You should take a backup (preferably automated) often - I have a scritp set to do this every week - but you should also backup before and after any major changes that you make.

Sorting: Selection Sort

Since I seem to be doing quite a few sorting algorithsms in my lectures at the moment, it would appear that I will be doing a series of sorting algorithm posts.

Today I did not have all that much time, so I wrote this one in javascript. I present to you: The selection sort.

The selection sort is a sorting algorithm where the smallest number is found and swapped with the number at the beginning, then the next smallest is swaped with the one next to the smallest number, and so on until the whoel sequence has been sorted.

Wikipedia has a rather nice animated GIF that explains the selection sort well here.

My implementation could be made faster be doing it in reverse, but I didn't have time to reverse it at the time of writing this post (check the comments to see if I did it later).

function selectionsort(arr)
{
    var maxid = 0, temp;
    for(var nextid = 0; nextid < arr.length - 1; nextid++)
    {
        //find the next smallest number remaining
        maxid = nextid;
        for(var j = nextid; j < arr.length; j++)
        {
            /*
            Invert to |
            sort desc.V */
            if(arr[j] < arr[maxid])
                maxid = j;
        }
        //swap the next number into place
        arr[nextid] = arr.splice(maxid, 1, arr[nextid])[0];

        console.log("nextid:", nextid, "state:", arr);
    }

    return arr; //for chaining
}

Here is a minified version (146 chars including newlines):

function selectionsort(e){for(var t=0,n=0;n<e.length-1;n++){t=n
for(var l=n;l<e.length;l++)e[l]<e[t]&&(t=l)
e[n]=e.splice(t,1,e[n])[0]}return e}

This was minified with UglifyJS.

Sample Output:

original [ 75, 4, 32, 87, 69, 73, 10, 18, 48, 2, 48, 96, 75, 36, 26 ]
nextid: 0 state: [ 2, 4, 32, 87, 69, 73, 10, 18, 48, 75, 48, 96, 75, 36, 26 ]
nextid: 1 state: [ 2, 4, 32, 87, 69, 73, 10, 18, 48, 75, 48, 96, 75, 36, 26 ]
nextid: 2 state: [ 2, 4, 10, 87, 69, 73, 32, 18, 48, 75, 48, 96, 75, 36, 26 ]
nextid: 3 state: [ 2, 4, 10, 18, 69, 73, 32, 87, 48, 75, 48, 96, 75, 36, 26 ]
nextid: 4 state: [ 2, 4, 10, 18, 26, 73, 32, 87, 48, 75, 48, 96, 75, 36, 69 ]
nextid: 5 state: [ 2, 4, 10, 18, 26, 32, 73, 87, 48, 75, 48, 96, 75, 36, 69 ]
nextid: 6 state: [ 2, 4, 10, 18, 26, 32, 36, 87, 48, 75, 48, 96, 75, 73, 69 ]
nextid: 7 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 87, 75, 48, 96, 75, 73, 69 ]
nextid: 8 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 75, 87, 96, 75, 73, 69 ]
nextid: 9 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 69, 87, 96, 75, 73, 75 ]
nextid: 10 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 69, 73, 96, 75, 87, 75 ]
nextid: 11 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 69, 73, 75, 96, 87, 75 ]
nextid: 12 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 69, 73, 75, 75, 87, 96 ]
nextid: 13 state: [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 69, 73, 75, 75, 87, 96 ]
result [ 2, 4, 10, 18, 26, 32, 36, 48, 48, 69, 73, 75, 75, 87, 96 ]

At some point I may port this to C♯.

(Probably) Coming Soon: (Binary) Insertion sort, Merge Sort, Quick Sort.

Security update to atom.gen.php

Since this website gets a lot of spam (ongoing investigations are currently in force in order to analyse the spambots' patterns, a post will be made here when they have been stopped) and this website also has a comments feed powered by atom.gen.php, I have had a chance to test atom.gen.php out in the wild with real data.

I discovered, unfortunately, that the script didn't handle invalid utf-8 and non printable characters very well, and this lead to the feed getting broken because XML doesn't like certain specific characters. This has now been fixed.

If you handle user input and use atom.gen.php to turn it into a feed, you will want to grab an updated copy of the script (quick link here) and overwrite your previous copy in order to fix this.

As well as fixing that, I also added a new option, $usecdata. This controls whether the <content> tag's contents should be wrapped in <![CDATA[...]]>. This should add extra protection again html / javascript injection attacks breaking your feeds. It defaults to false, though, so you need to manually enable it by setting it to true.

The reference has been updated accordingly.

If you find another bug, please comment below. You will recieve full credit at the top of the file (especially if you provide a fix!).

Reverse Bubble Sorting

We had our second algorithms lecture this week - this time is was on bubble sorting. Apparently we will be doing several sorting algorithms over the next few weeks, each with their own strengths and weaknesses.

Today I bring you an optimised bubble sort implementation in C♯! This will be the first C♯ code that I have posted on this blog.

Basically, the bubble sort algorithm iterates over an array of numbers repeatedly and swaps those that are in the wrong order, until there aren't any more numbers left to swap.

Here is the script:

static void DoBubbleSort(int[]arraytosort) {
    int endingpoint = 1,
        temp;
    bool issorted;

    int swaps = 0,
        iterations = 0,
        passes = 0; //debug

    do {
        issorted = true;

        for(int i = arraytosort.Length - 1; i >= endingpoint; i--)
        {
            iterations++; //debug
            //Console.WriteLine("i: " + i + " i-1: " + (i - 1));

            if (arraytosort[i - 1] > arraytosort[i])
            {
                swaps++; //debug
                //swap the numbers around
                temp = arraytosort[i - 1];
                arraytosort[i - 1] = arraytosort[i];
                arraytosort[i] = temp;

                issorted = false;
            }
        }

        Console.Write("pass: " + passes + " ");
        printarray(arraytosort); //debug
        Console.WriteLine();

        passes++; //debug

        endingpoint++;
    } while (!issorted);

    Console.WriteLine("Sorting Complete!");
    Console.WriteLine("Statistics\n----------");
    Console.WriteLine("Passes: " + passes + ", Iterations: " + iterations + " Swaps: " + swaps);
}

...and here is an example of what it outputs:

Original: [66, 51,  0,  5, 42, 92,  8,  8, 28,  8]
pass: 0 [ 0, 66, 51,  5,  8, 42, 92,  8,  8, 28]
pass: 1 [ 0,  5, 66, 51,  8,  8, 42, 92,  8, 28]
pass: 2 [ 0,  5,  8, 66, 51,  8,  8, 42, 92, 28]
pass: 3 [ 0,  5,  8,  8, 66, 51,  8, 28, 42, 92]
pass: 4 [ 0,  5,  8,  8,  8, 66, 51, 28, 42, 92]
pass: 5 [ 0,  5,  8,  8,  8, 28, 66, 51, 42, 92]
pass: 6 [ 0,  5,  8,  8,  8, 28, 42, 66, 51, 92]
pass: 7 [ 0,  5,  8,  8,  8, 28, 42, 51, 66, 92]
pass: 8 [ 0,  5,  8,  8,  8, 28, 42, 51, 66, 92]
Sorting Complete!
Statistics
----------
Passes: 9, Iterations: 45 Swaps: 24

The script keeps track of the furthest point in the array it reached on each pass and goes one less each time - this is because the smallest number will always get pushed into its proper place at the left hand side on each pass.

As for the reason the script iterates backwards, in Javascript it is recommended that you iterate backwards to avoid repeatedly referencing Array.length, since it has to count the contents of an array upon each refernce. This is probably not the case with C♯, but it is a habit of mine :)

It is important to note that even though the function doesn't return anything, it still sorts the array because arrays are passed by reference by default, just like in Javascript (aka Ecmascript).

There are quite a few debug statements in there. Remove then for actual use in your code.

A (64 bit) compiled version of the script is available:

reversebubblesort.exe

Hashes:

Algorithm Hash
CRC32 644f6c6a
MD5 93fba7a072954ee6f34fcf44913eadc7
SHA1 01a54b24c475ec2ff1bf159dc1224e10553f430d
SHA-256 d32d689e2785d738c54e43a9dc70c1d8f2de76383022a87aa4f408519a7941cb
SHA-384 df7c4ac441aabaa1f182ade7532885d8ee5518c26f17d72d7952dcfaa39552dda9ad219a37661591fea169fd6ed514bb
SHA-512 c993509901bb65cd893d1c8455c5ad8dc670632e5476aad899980348b45bc3435cfab3fe6d8fd80606cfea3608770c9900be51e09f6f1a8c9fd5fe28169fd81d

Remember to always verify the integrity of your downloaded files, especially the larger ones. If you would like another type of binary (e.g. 32 bit, ARM, etc.), please post a comment below and I will reply with a download link. The compiler used was csc.exe on a Windows 7 64 bit command line.

Questions and / or comments are welcome below.

Art by Mythdael