Starbeamrainbowlabs

Stardust
Blog


Archive

Mailing List Articles Atom Feed Comments Atom Feed Twitter

Tag Cloud

3d account algorithms announcement archives arduino artificial intelligence assembly async audio bash batch blog bookmarklet booting c sharp c++ challenge chrome os code codepen coding conundrums coding conundrums evolved command line compiling css dailyprogrammer debugging demystification distributed computing downtime embedded systems encryption es6 features event experiment external first impressions future game github github gist graphics hardware hardware meetup holiday html html5 html5 canvas interfaces internet io.js jabber javascript js bin labs learning library linux low level lua maintenance network networking node.js operating systems performance photos php pixelbot portable privacy programming problems project projects prolog protocol protocols pseudo 3d python reddit reference release releases resource review rust secrets security series list server software sorting source code control statistics svg technical terminal textures three thing game three.js tool tutorial tutorials twitter ubuntu university update updates upgrade version control visual web website windows windows 10 xmpp

Profiling PHP with XDebug

(This post is a fork of a draft version of a tutorial / guide originally written as a whilst at my internship.)

The PHP and xdebug logos.

Since I've been looking into xdebug's profiling function recently, I've just been tasked with writing up a guide on how to set it up and use it, from start to finish - and I thought I'd share it here.

While I've written about xdebug before in my An easier way to debug PHP post, I didn't end up covering the profiling function - I had difficulty getting it to work properly. I've managed to get it working now - this post documents how I did it. While this is written for a standard Debian server, the instructions can easily be applied to other servers.

For the uninitiated, xdebug is an extension to PHP that aids in the debugging of PHP code. It consists of 2 parts: The php extension on the server, and a client built into your editor. With these 2 parts, you can create breakpoints, step through code and more - though these functions are not the focus of this post.

To start off, you need to install xdebug. SSH into your web server with a sudo-capable account (or just use root, though that's bad practice!), and run the following command:

sudo apt install php-debug

Windows users will need to download it from here and put it in their PHP extension direction. Users of other linux distributions and windows may need to enable xdebug in their php.ini file manually (windows users will need extension=xdebug.dll; linux systems use extension=xdebug.so instead).

Once done, xdebug should be loaded and working correctly. You can verify this by looking the php information page. To see this page, put the following in a php file and request it in your browser:

<?php
phpinfo();
?>

If it's been enabled correctly, you should see something like this somewhere on the resulting page:

Part of the php information page, showing the xdebug section.

With xdebug setup, we can now begin configuring it. Xdebug gets configured in php.ini, PHP's main configuration file. Under Virtualmin each user has their own php.ini because PHP is loaded via CGI, and it's usually located at ~/etc/php.ini. To find it on your system, check the php information page as described above - there should be a row with the name "Loaded Configuration File":

The 'loaded configuration file' directive on the php information page.

Once you've located your php.ini file, open it in your favourite editor (or type sensible-editor php.ini if you want to edit over SSH), and put something like this at the bottom:

[xdebug]
xdebug.remote_enable=1
xdebug.remote_connect_back=1
xdebug.remote_port=9000
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_autostart=true

xdebug.profiler_enable=false
xdebug.profiler_enable_trigger=true
xdebug.profiler_enable_trigger_value=ZaoEtlWj50cWbBOCcbtlba04Fj
xdebug.profiler_output_dir=/tmp
xdebug.profiler_output_name=php.profile.%p-%u

Obviously, you'll want to customise the above. The xdebug.profiler_enable_trigger_value directive defines a secret key we'll use later to turn profiling on. If nothing else, make sure you change this! Profiling slows everything down a lot, and could easily bring your whole server down if this secret key falls into the wrong hands (that said, simply having xdebug loaded in the first place slows things down too, even if you're not using it - so you may want to set up a separate server for development work that has xdebug installed if you haven't already). If you're not sure on what to set it to, here's a bit of bash I used to generate my random password:

dd if=/dev/urandom bs=8 count=4 status=none | base64 |  tr -d '=' | tr '+/' '-_'

The xdebug.profiler_output_dir lets you change the folder that xdebug saves the profiling output files to - make sure that the folder you specify here is writable by the user that PHP is executing as. If you've got a lot of profiling to do, you may want to consider changing the output filename, since xdebug uses a rather unhelpful filename by default. The property you want to change here is xdebug.profiler_output_name - and it supports a number of special % substitutions, which are documented here. I can recommend something phpprofile.%t-%u.%p-%H.%R.%cachegrind - it includes a timestamp and the request uri for identification purposes, while still sorting chronologically. Remember that xdebug will overwrite the output file if you don't include something that differentiates it from request to request!

With the configuration done, we can now move on to actually profiling something :D This is actually quite simple. Simply add the XDEBUG_PROFILE GET (or POST!) parameter to the url that you want to test in your browser. Here are some examples:

https://localhost/carrots/moon-iter.php?XDEBUG_PROFILE=ZaoEtlWj50cWbBOCcbtlba04Fj
https://development.galacticaubergine.de/register?vegetable=yes&mode=plus&XDEBUG_PROFILE=ZaoEtlWj50cWbBOCcbtlba04Fj

Adding this parameter to a request will cause xdebug to profile that request, and spit out a cachegrind file according to the settings we configured above. This file can then be analysed in your favourite editor, or, if it doesn't have support, an external program like qcachegrind (Windows) or kcachegrind (Everyone else).

If you need to profile just a single AJAX request or similar, most browsers' developer tools let you copy a request as a curl or wget command (Chromium-based browsers, Firefox - has an 'edit and resend' option), allowing you to resend the request with the XDEBUG_PROFILE GET parameter.

If you need to profile everything - including all subrequests (only those that pass through PHP, of course) - then you can set the XDEBUG_PROFILE parameter as a cookie instead, and it will cause profiling to be enabled for everything on the domain you set it on. Here's a [bookmarklet]() that set the cookie:

javascript:(function(){document.cookie='XDEBUG_PROFILE='+'insert_secret_key_here'+';expires=Mon, 05 Jul 2100 00:00:00 GMT;path=/;';})();

(Source)

Replace insert_secret_key_here with the secret key you created for the xdebug.profiler_enable_trigger_value property in your php.ini file above, create a new bookmark in your browser, paste it in (making sure that your browser doesn't auto-remove the javascript: at the beginning), and then click on it when you want to enable profiling.

Was this helpful? Got any questions? Let me know in the comments below!

Sources and further reading

An easier way to debug PHP

A nice view of some trees taken by Mythdael I think, the awesome person who designed this website :D

Recently at my internship I've been writing quite a bit of PHP. The language itself is OK (I mean it does the job), but it's beginning to feel like a relic of a bygone era - especially when it comes to debugging. Up until recently I've been stuck with using echo() and var_dump() calls all over the place in order to figure out what's going on in my code - that's the equivalent of debugging your C♯ ACW with Console.WriteLine() O.o

Thankfully, whilst looking for an alternative, I found xdebug. Xdebug is like visual studio's debugging tools for C♯ (albeit a more primitive form). They allow you to add breakpoints and step though your PHP code one line at a time - inspecting the contents of variables in both the local and global scope as you go. It improves the standard error messages generated by PHP, too - adding stack traces and colour to the output in order to make it much more readable.

Best of all, I found a plugin for my primary web development editor atom. It's got some simple (ish) instructions on how to set up xdebug too - it didn't take me long to figure out how to put it to use.

I'll assume you've got PHP and Nginx already installed and configured, but this tutorial looks good (just skip over the MySQL section) if you haven't yet got it installed. This should work for other web servers and configurations too, but make sure you know where your php.ini lives.

XDebug consists of 2 components: The PHP extension for the server, and the client that's built into your editor. Firstly, you need to install the server extension. I've recorded an asciicast (terminal recording) to demonstrate the process:

(Above: An asciinema recording demonstrating how to install xdebug. Can't see it? Try viewing it on asciinema.org.)

After that's done, you should be able to simply install the client for your editor (I use php-debug for atom personally), add a breakpoint, load a php page in your web browser, and start debugging!

If you're having trouble, make sure that your server can talk directly to your local development machine. If you're sitting behind any routers or firewalls, make sure they're configured to allow traffic though on port 9000 and configured to forward it on to your machine.

TeleConsole: A simple remote debugging solution for awkward situations

Header image (Header image from here.)

Several times in the last few months I've found myself in some kind of awkward situation where I need to debug a C♯ program, but the program in question either doesn't have a console, or is on a remote machine. In an ideal world, I'd like to have all my debugging message sent to my development machine for inspection. That way I don't have to check the console of multiple different machines just to get an idea as to what has gone wrong.

C♯ already has System.Diagnostics.Debug, which functions similarly to the Console class, except that it sends data to the Application output window. This is brilliant for things running on your local machine through Visual Studio or MonoDevelop, but not so great when you've got a program that utilises the network and has to run on separate computers. Visual Studio for one starts to freeze up if you open the exact same repository on a network drive that's already open somewhere else.

It is for these reasons that I finally decided to sit down and write TeleConsole. It's a simple remote console that you can use in any project you like. It's split into 2 parts: the client library and the server binary. The server runs on your development machine, and listens for connections from the client library, which you reference in your latest and greatest project and use to send messages to the server.

Take a look here: sbrl/TeleConsole (GitHub) (Direct link to the latest release)

The client API is fully documented with intellisense comments, so I assume it should be very easy to work out how to use it (if there's something I need to do to let you use the intellisense comments when writing your own programs, let me know!). If you need some code to look at, I've created an example whilst testing it.

Although it's certainly not done yet, I'll be definitely be building upon it in the future to improve it and add additional features.

Test C♯ code online with repl.it

I've known about repl.it for a while now. It is a site that provides you with a REPL (Read-Eval-Print-Loop) for many different languages, without you having to install the language in question thanks to the native client.

A REPL (in case you didn't know) is like a command prompt, but for a specific programming language or environment. For example, if you type node into your command prompt (if you have Node.js installed), it will start a REPL for you to play around with.

Recently I have discovered that repl.it also supports C♯ (via the mono compiler version 4.0.4.0 at the time of typing), and it lets you write, compile and run C♯ code without ever leaving your browser. I was so surprised by this I thought that I'd make a blog post about it. Apparently you can even embed things you've created into other pages too - here's a small test program I wrote whilst playing around with it:

Links:

Update: Corrected The expansion of REPL.

Reading HTTP 1.1 requests from a real web server in C#

I've received rather a lot of questions recently asking the same question, so I thought that I 'd write a blog post on it. Here's the question:

Why does my network client fail to connect when it is using HTTP/1.1?

I encountered this same problem, and after half an hour of debugging I found the problem: It wasn't failing to connect at all, rather it was failing to read the response from the server. Consider the following program:

using System;
using System.IO;
using System.Net.Sockets;

class Program
{
    static void Main(string[] args)
    {
        TcpClient client = new TcpClient("host.name", 80);
        client.SendTimeout = 3000;
        client.ReceiveTimeout = 3000;
        StreamWriter writer = new StreamWriter(client.GetStream());
        StreamReader reader = new StreamReader(client.GetStream());
        writer.WriteLine("GET /path HTTP/1.1");
        writer.WriteLine("Host: server.name");
        writer.WriteLine();
        writer.Flush();

        string response = reader.ReadToEnd();

        Console.WriteLine("Got Response: '{0}'", response);
    }
}

If you change the hostname and request path, and then compile and run it, you (might) get the following error:

An unhandled exception of type 'System.IO.IOException' occurred in System.dll

Additional information: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time or established connection failed because connected host has failed to respond.

Strange. I'm sure that we sent the request. Let's try reading the response line by line:

string response = string.Empty;
do
{
    string nextLine = reader.ReadLine();
    response += nextLine;
    Console.WriteLine("> {0}", nextLine);
} while (reader.Peek() != -1);

Here's some example output from my server:

> HTTP/1.1 200 OK
> Server: nginx/1.9.10
> Date: Tue, 09 Feb 2016 15:48:31 GMT
> Content-Type: text/html
> Transfer-Encoding: chunked
> Connection: keep-alive
> Vary: Accept-Encoding
> strict-transport-security: max-age=31536000;
>
> 2ef
> <html>
> <head><title>Index of /libraries/</title></head>
> <body bgcolor="white">
> <h1>Index of /libraries/</h1><hr><pre><a href="../">../</a>
> <a href="prism-grammars/">prism-grammars/</a>
   09-Feb-2016 13:56                   -
> <a href="blazy.js">blazy.js</a>                                           09-F
eb-2016 13:38                9750
> <a href="prism.css">prism.css</a>                                          09-
Feb-2016 13:58               11937
> <a href="prism.js">prism.js</a>                                           09-F
eb-2016 13:58               35218
> <a href="smoothscroll.js">smoothscroll.js</a>
   20-Apr-2015 17:01                3240
> </pre><hr></body>
> </html>
>
> 0
>

...but we still get the same error. Why? The reason is that the web server is keeping the connection open, just in case we want to send another request. While this would usually be helpful (say in the case of a web browser - it will probably want to download some images or something after receiving the initial response), it's rather a nuisance for us, since we don't want to send another request and it's rather awkward to detect the end of the response without detecting the end of the stream (that's what the while (reader.Peek() != -1); is for in the example above).

Thankfully, there are a few solutions to this. Firstly, the web server will sometimes (but not always - take the example response above for starters) send a content-length header. This header will tell you how many bytes follow after the double newline (\r\n\r\n) that separate the response headers from the response body. We could use this to detect the end of the message. This is the reccommended way , according to RFC2616.

Another way to cheat here is to send the connection: close header. This instructs the web server to close the connection after sending the message (Note that this will break some of the tests in the ACW, so don't use this method!). Then we can use reader.ReadToEnd() as normal.

A further cheat would be to detect the expected end of the message that we are looking for. For HTML this will practically always be </html>. We can close the connection after we receive this line (although this doesn't work when you're not receiving HTML). This is seriously not a good idea. The HTML could be malformed, and not contain </html>.

Prolog Visualisation Tool

A visualisation of depth first search in Prolog.

Recently, I've been finding that Prolog is getting rather more complicated, and that the traces that I keep doing are getting longer and longer. This is making it rather difficult to understand what's going on, and so in response to this I am building the Prolog Visualisation Tool(kit).

Basically, the Prolog Visualisation Tool(kit) is a tool that, given a Prolog trace, produces a diagram of the trace in question. The image at the top of this post is diagram produced by the tool for a depth first search.

You can find it live now on GitHub Pages.

It is built with mermaid, a really cool diagramming library by knsv, which converts some custom graph syntax to an svg.

The next step will be to animate it, but I haven't got that far yet. Expect an update soon!

Art by Mythdael