Mailing List
Articles Atom Feed
Comments Atom Feed
Twitter
Reddit
Facebook
Recently the order of the posts on this blog seemed to go all strange - I have spent some time attempting to fix it. Hopefully things are back to normal now. Because of this, this week's ES6: Features post will be released tomorrow.
I now have a new tool that re-orders the posts on this blog - problems like this one shouldn't happen (as often!) in the future.
Welcome to the 9th postin my weekly ES6: Features series. Today, I will be taking a look at Map, and its counterpart, the WeakMap.
At the simplest level, a Map in Javascript is a set of key-value pairs. You can do things like this:
> var map = new Map();
undefined
> map.set("Sticky Swamp", { x: 5, y: 7 });
Map { 'Sticky Swamp' => { x: 5, y: 7 } }
> map.set("Frosty Mountain", { x: 3, y: 4 });
Map {
'Sticky Swamp' => { x: 5, y: 7 },
'Frosty Mountain' => { x: 3, y: 4 } }
> map.set("Piranha Pond", { x: 7, y: 2 });
Map {
'Sticky Swamp' => { x: 5, y: 7 },
'Frosty Mountain' => { x: 3, y: 4 },
'Piranha Pond' => { x: 7, y: 2 } }
> map.get("Frosty Mountain").x
3
> map.has("Treasure")
false
> map.has("Piranha Pond")
trueMaps can also be iterated. With a bit of destructuring wizardry, we can list the content of a map like this:
> for(var [ key, value ] of map) { console.log(`Name: ${key} Location: (${value.x}, ${value.y})`); }
Name: Sticky Swamp Location: (5, 7)
Name: Frosty Mountain Location: (3, 4)
Name: Piranha Pond Location: (7, 2If you only want to iterate over the keys or the values, you can get an iterable object of them with .keys() and .values() respectively.
> for(var key of map.keys()) { console.log(key); }
Sticky Swamp
Frosty Mountain
Piranha Pond
undefined
> for(var key of map.values()) { console.log(key); }
{ x: 5, y: 7 }
{ x: 3, y: 4 }
{ x: 7, y: 2 }
undefinedNote that you can't use .forEach() or any related functions (such as .map() or .filter()) on this object, and it's contents doesn't show up in the developer tools either.
Another cool feature of Maps is that you can use anything is the key. This allows you to create empty objects (for example), like so:
> map = new Map();
Map {}
> a = {}
{}
> b = {}
{}
> a == b
false
> map.set(a, "this is a")
Map { {} => 'this is a' }
> map.set(b, "this is b");
Map { {} => 'this is a', {} => 'this is b' }
> map.get(a)
'this is a'
> map.get(b)
'this is b'This probably won't make too much sense with a normal Map, but mixing this with the WeakMap leads to some powerful data storage mechanisms.
WeakMapsThe WeakMap is the cousin of the Map, and is largely the same, but with a few important differences. Firstly, it isn't iterable, and you can't get a list a keys from it either. If you need sa list of keys, you should maintain it yourself.
This means that if you are using objects as your keys (as described above), and you loose all the references to that object, the garbage collector can not only come and collect the object you lost all references to, but also the value that it was tied to in the WeakMap. This helps you to avoid memory leaks if you don't actually need to iterate over the map at any point.
> wmap = new WeakMap()
{}
> a = {};
{}
> b = {};
{}
> wmap.set(a, "This is a")
{}
> wmap.set(b, "This is b");
{}
> wmap.get(a)
'This is a'
> wmap.get(b)
'This is b'
> delete a
true
> wmap.get(b)
'This is b'
> wmap.get(a)
ReferenceError: a is not definedIn the example above, once we delete a, we also loose our only connection with the data that it was tied to in the WeakMap, allowing the garbage collector to come over and clean thing up for us.
Other than that, the WeakMap acts much the same as the Map - it has the same get(), set() and has() methods that the Map does.
That concludes the 9th post in this series (sorry it was rather late)! Next time, I will probably take a look at Set.
A little while ago, I wrote my first distributed computing program to solve the second optional challenge to /r/dailyprogrammer's #223rd problem. I wrote it in Javascript, because I know really know how to utilise the tcp/ip networking stack in any other language yet :(
Anyway, if you want to check out my source code, I have put it up on GitHub. Below I will explain how you can get started with the pair for scripts I wrote, and how they work
Getting started is easy. Just clone the repository & cd into the new directory:
git clone https://github.com/sbrl/dailyprogrammer-223.git
cd dailyprogrammer-223Then install the dependency:
npm installThen you can start a server like so:
node server.js 4321 # starts a new server on port 4321Or you can start a client like this:
node client.js starbeamrainbowlabs.com:9999Note that I have used some ES6 features (check out my ES6: Features series), so if you are using Node.js and not io.js, you will need to tack on the --harmony flag in order to get them to work.
It works by mapping the search space (aaaaa to zzzzz) to a set of numbers, starting at 0. I used an ES6 generator to keep track of where we have got up to, wrapped in a function that handles reallocating work units that haven't been completed within a certain time limit. Each work unit consist of 16 words by default (though it can be changed), with the time limit set to 5 times the number of words in the block.
Next time, I will write the client so that it is compatible with a conventional browser - this should make it much easier for people to contribute their CPU time!
This week's ES6 post is about a rather strange (but useful) feature that I have only just discovered: Symbols. Before I explain to you how they work, be warned: I don't understand them fully myself yet.
Symbols are like unique keys and can be used as properties so you can use to ensure that you don't clash with any other property in an object. When creating one, you can optionally give it a 'name' so you know what it is - this can be any data type, but Symbol() will call .toString() on it first. Note that two symbols created with the same name aren't the same:
> sym1 = Symbol("starbeamrainbowlabs")
Symbol(starbeamrainbowlabs)
> sym2 = Symbol("starbeamrainbowlabs")
Symbol(starbeamrainbowlabs)
> sym1 == sym2
falseStrange, right?! This can be very useful though. You could use this to attach extra data to an object that you have been given by a library to identify it, for example, or you can use it to store something in a browser's localStorage without clashing with any other scripts running on the same domain (but this apparently doesn't persist across page reloads):
> localStorage[Symbol("bobs-rockets")] = "Some data";
"Some data"
> localStorage[Symbol("bobs-rockets")] = "Some data";
"Some data"
> localStorage
Storage { Symbol(bobs-rockets): "Some data", Symbol(bobs-rockets): "Some data" }While this is cool, this raises the issue of retrieving the data we stored when we next load the page. If we try to create a new symbol with the same name that we used before, the browser returns undefined because the symbol we just created is not the same as the one we created the last time we loaded the page, as we discovered above.
Our next thought might be to iterate over the object and try to work out which one is ours:
> Object.keys(data).forEach(function(key) {
... console.log(`${key}: ${value}`);
... });
some-key: some-data
some-other-key: some-other-data
undefinedBut weirdly, the Symbols we used don't show up! Object.keys() only returns an array of string based keys, and completely ignores the Symbols we created. This calls for a special function: Object.getOwnPropertySymbols(). This returns an array of all the Symbol based properties in an object. Then you can iterate over them like so:
> Object.getOwnPropertySymbols(data).forEach(function(key) {
... console.log(`${key.toString()}: ${value}`);
... });
Symbol(bobs-rockets): some-data
Symbol(bobs-rockets): some-data
undefinedNote the .toString() call here. If you try and convert a Symbol to a string, an exception is thrown:
TypeError: Cannot convert a Symbol value to a stringThe same goes for if you accidentally use the new keyword when creating a Symbol:
> var sym = new Symbol();
TypeError: Symbol is not a constructorThere is an ever better way to recreate the same symbol across page loads though. You can use the Symbol.for() function:
> bobsSymbol = Symbol.for("bobs-rockets")
Symbol(bobs-rockets)
> bobsSymbol2 = Symbol.for("bobs-rockets")
Symbol(bobs-rockets)
> bobsSymbol == bobsSymbol2
true
> Symbol.for("bobs-rockets") == Symbol.for("bills-boosters")
falseThis also allows two libraries to share the same symbol - but make sure that the string you pass to Symbol.for() is unique. This brings me onto the last thing I found out - you can reverse engineer a Symbol you obtained through Symbol.for() to work out the string used to create it with Symbol.keyFor():
> bobsSymbol = Symbol.for("bobs-rockets")
Symbol(bobs-rockets)
> Symbol.keyFor(bobsSymbol)
'bobs-rockets'That concludes this ES6: Features post. To summarise, Symbols are unique keys that can be used to avoid conflicts in property names of objects. Every time you create one, you make a brand new Symbol that is completely different to any created before, except if Symbol.for() was used. Symbols created with Symbol.for() can be reverse-engineered to extract the string used to create it. Next time, I will probably look at well known symbols.
This post is slightly different - It is mainly to document the process of creating a flash drive that boots into Grub2 with both a BIOS and UEFI firmware that you can also store data on in Windows.
First of all, you need to make sure that your flash drive has a GPT (GUID Partition Table) and not an MBR. If you don't know, it is safe to assume that you are currently using a MBR (Master Boot Record).
If you don't have a GPT, then you need to backup the contents of your flash drive because you will need to wipe it clean in order to switch it over.
Make sure you are in a linux environment (start a virtual machine and pass the flash drive in if you don't have one). Work out where your flash drive is (in this instance ours is at /dev/sdx) and then execute the following commands:
sudo apt-get install gdisk
sudo sgdisk --zap-all /dev/sdx
The above will wipe all partition tables from the device. Before continuing, you may need to remove and re-plug-in the flash drive you are working in. Run this pair of commands to add a "Microsoft basic data" partition, and format it with FAT32:
sudo sgdisk --new=1:0:0 --typecode=1:0700 /dev/sdx
sudo mkfs.vfat -F32 -n GRUB2EFI /dev/sdx1
The type code 0700 is the bit that determines the type of partition that we are creating. In this case, we are creating a Microsoft basic data partition. The tutorial I followed (see the source at the bottom) set the type ef00, which is a EFI System Partition. If you experience problems, try changing the type to this.
Next up, we need to mount the new partition. Do it like this:
sudo mount -t vfat /dev/sdb1 /mnt -o uid=1000,gid=1000,umask=022
The above will mount it to the directory /mnt. next, you need to go to the first source below and download the zip that can be found under the text pack with all necessary files for you to modify as you need. Extract this to the root of the flash drive (/mnt in our case):
cd ~/Downloads/
unzip usb-pack_efi.zip
rsync -auv usb-pack_efi/ /mnt
Next, we need to install grub2 in BIOS mode. It will complain horribly, but apparently it works :D
sudo grub-install --force --removable --boot-directory=/mnt/boot /dev/sdb
Now you should have a flash drive with Grub2 installed, that you can also see in Windows!
Credit to sysmatck of ubuntuforums.org sudodus for the guide.
Sources:
This week's ES6 Features post is going to be on some rather new syntactic sugar, called destructuring. Please be aware that neither Node.js or io.js support this syntax, and the only browser to have a decent level of support is Firefox.
Destructuring allows you to take an array's contents and assign it's values to multiple variables at once. Whereas before you would probably do this:
> var locations = [ "Paris", "New York" ];
undefined
> var first = locations[0], second = locations[1];
undefined
> first
'Paris'
> second
'New York'You can now do this:
> var locations = [ "Paris", "New York" ];
undefined
> var [ first, second ] = locations;
undefined
> first
'Paris'
> second
'New York'Destructuring also supports the rest operator (...), so you can assign the rest of the values in an array to a variable too:
> var fruits = [ "orange", "apple", "banana", "kiwi", "avocado" ];
undefined
> one
"orange"
> two
"apple"
> var [one,two,three,...rest] = fruits
undefined
> three
"banana"
> rest
["kiwi", "avocado"]Destructuring in Ecmascript 7 can also be used on objects too, allowing you to set multiple variables equal to the value of a property in any object:
> var item = { "name": "compass", "quantity": 1, "value": 250 };
undefined
> var {name, quantity, value} = item;
undefined
> name
'compass'
> quantity
1
> value
250In order for object destructuring to work, the variable names you are declaring must have the same name as the property of the object that you want to destructure.
Destructuring also works in a function context too! If you specify an object as the only parameter to a function, you can destructure it's contents into multiple variables. This looks like a good way to simplify complex functions that have a lot of arguments.
> function test({ x: x, y: y }) { return `(${x}, ${y})`; }
undefined
> test({ x: 250, y: 50 });
'(250, 50)'That concludes this ES6: Features post on destructuring. To summarise, destructuring is a way simplify the assigning of multiple variables at once to the values in an array. The rest operator can be used to gather up the rest of the elements left in an array. Objects can be destructured too, by both variable assignments and the arguments of a function.
Destructuring isn't generally available yet though. Just before I end this post, here's a table of who supports it:
| Environment | Support? |
|---|---|
| Internet Explorer 11 | No |
| Microsoft Edge | No |
| Chrome 44 | No |
| Firefox 39 | Yes |
| Opera Beta 31 | No |
| io.js v2.3.1 | No |
| Node.js 0.12 | No |
| Babel | Yes |
We have reached the 6th post in the ES6 features series! This series of posts has to be my longest yet. This week's post is a short one about a new keyword that has been added in ES6: const. If you know any C♯, then this keywrod may already be familiar to you. It's purpose is to define a variable (called a constant) that cannot be changed or redefined.
As with normal variables, constants are function scope. When defined outside a function, they become part of the global scope. Example:
> const max_value = 64;
undefined
> max_value
64
Since constants can't be changed, setting them to a different value has no effect (but doesn't throw an error - wait, what?):
> max_value = 128;
128
> max_value
64
Redefining a constant throws an error:
> max_value
64
> const max_value = 128
TypeError: Identifier 'max_value' has already been declared
And they can't be deleted either:
> delete max_value
false
> max_value
64
It seems that you can't define a constant as a property of an object though (makes sense - then it would be a property and not a variable I suppose, though having the the syntax work like that would make things so much more readable). For that, you'll have to use Object.defineProperty() with the writable: false option.
> settings = {};
{}
> const settings.seed = 47264;
SyntaxError: Unexpected token .
Other than that, they behave like normal variables. For example you can convert a constant number to a string as normal:
> max_value.toString(2)
'1000000'
>Though curiously, mutators on constants work correctly and change the original variable (perhaps this is a bug in io.js?):
> const y = [1,2,3]
undefined
> y
[ 1, 2, 3 ]
> y.reverse()
[ 3, 2, 1 ]
> y
[ 3, 2, 1 ]
That concludes this experimentation-heavy post on constants. To summarise:
Hopefully this was useful to someone. Next time, I might take a look at some of the new additions to the Math object while I try and get my head around destructuring.
Recently I have been working on a website for someone I know. Mythdael (the awesome artist who created a design for this website!) did the design for this one too, and I felt that I had to bring it to life. While I was writing it, I found that I needed to convert any given hashtag into a presentable title. Since hashtags are all one word (and usually lower case), it is more or less impossible to work out where one word starts and another ends. The solution: A wordlist (My choice was a modified enable1.txt).
Here is the solution I came up with:
/*
* From https://terenceyim.wordpress.com/2011/02/01/all-purpose-binary-search-in-php/
* Parameters:
* $a - The sort array.
* $first - First index of the array to be searched (inclusive).
* $last - Last index of the array to be searched (exclusive).
* $key - The key to be searched for.
* $compare - A user defined function for comparison. Same definition as the one in usort
*
* Return:
* index of the search key if found, otherwise return (-insert_index - 1).
* insert_index is the index of smallest element that is greater than $key or sizeof($a) if $key
* is larger than all elements in the array.
*/
function binary_search(array $a, $first, $last, $key, $compare) {
$lo = $first;
$hi = $last - 1;
while ($lo <= $hi) {
$mid = (int)(($hi - $lo) / 2) + $lo;
$cmp = call_user_func($compare, $a[$mid], $key);
if ($cmp < 0) {
$lo = $mid + 1;
} elseif ($cmp > 0) {
$hi = $mid - 1;
} else {
return $mid;
}
}
return -($lo + 1);
}
class hashtag_parser
{
public $wordlist_length = 0;
public $wordlist = [];
function __construct($wordlist_path) {
global $settings;
$this->wordlist = file($wordlist_path, FILE_IGNORE_NEW_LINES);
$this->wordlist_length = count($this->wordlist);
}
public function in_wordlist($word)
{
$word = strtolower($word);
$result = binary_search($this->wordlist, 0, $this->wordlist_length, $word, "strcmp");
if($result > -1)
return true;
else
return false;
/*
if(in_array($word, $this->wordlist))
return true;
else
return false;
*/
}
public function extract_words($hashtag)
{
global $settings;
// Remove the hash from the beginning if it is present
if(substr($hashtag, 0, 1) == "#") $hashtag = substr($hashtag, 1);
// Create an array to hold the words we find
$words = [];
$length = strlen($hashtag); // Cache the length of the hashtag
$pos = 0;
while($pos < $length)
{
// echo("pos: $pos\n");
// aim: find the length of the longest substring that is a valid
// word according to the wordlist
$longest_word_length = 0;
for($scan_pos = $pos + 1; $scan_pos < $length + 1; $scan_pos++)
{
// echo("scan_pos: $scan_pos substring: " . substr($hashtag, $pos, $scan_pos - $pos) . "\n");
if($this->in_wordlist(substr($hashtag, $pos, $scan_pos - $pos)))
{
$longest_word_length = $scan_pos - $pos;
// echo("found word\n");
}
}
// Set the length of the longest word to the remainder of the
// string if we don't find any valid words
if($longest_word_length == 0) $longest_word_length = $length - $pos;
$words[] = substr($hashtag, $pos, $longest_word_length);
$pos += $longest_word_length;
}
return ucwords(implode(" ", $words));
}
}
The code is a bit messy (perhaps I should tidy it up a bit lot), but it does the job I intended it to do. I used a class because I was concerned that reading in the wordlist every time would cause the code to take too long to complete - it is slow enough as it is. Thankfully a binary search algorithm written in PHP by terenceyim helped speed things up enormously. Anyway, it can be used like this:
$parser = new hashtag_parser("/path/to/wordlist.txt");
echo($parser->extract_words("sometext")); // Prints "Some Text"The algorithm I used is quite simple:
I am posting this here in the hopes that someone else will find this code useful :)
Unfortunately I have been rather busy this week so far, but I managed to find time to write up another post for you. This week's ES6 features are spread and rest. They go hand in hand, so I thought I would cover them both at once.
The spread operator lets you spread an array's contents out as the arguments when calling a function. It works like Function.apply, but it looks much neater :)
function multiply(a, b, c)
{
return a * b * c;
}
var numbers = [4, 7, 2];
console.log(multiply(...numbers)); // 56Before you might have had to do something like this:
console.log(multiply.apply(this, numbers)); // 56Personally, I think that the new addition make a lot more sense than the old apply function, although you will still need to use Function.apply() if you want to customise the execution context of the function.
The spread operator, as you might expect, spread an array of things out to different arguments. But what if you wanted to bunch them back up again? ES6 can handle that too, through the rest operator. The rest operator allows you to bunch the rest of the arguments passed to a function into an array:
function log_adv(who, ...what)
{
"use strict";
var str = `[ ${new Date().toLocaleString()} ] [ ${who} ] `;
for(let thing of what)
{
str += JSON.stringify(thing) + " ";
}
console.log(str);
}
var x = Math.random(),
y = Math.random() * 2;
// Logs something like:
// [ 08/07/2015, 11:51:14 ] [ program ] "x is" 0.051463941344991326 "y is" 1.5026674889959395
log_adv("program", "x is", x, "y is", y);So there you have it! Two more new features of ES6. Next time I might look into arrow functions. Also, this Friday I will be posting about a PHP hashtag-to-title converter that I wrote for a project that I am working on for someone I know.
Hello again. This is just a quick post to let you know that I am hosting the enable1.txt wordlist on this site. The only copy I could find on the internet is on Google Code, which is shutting down.
I don't know where this wordlist originated (leave a comment below if you know!), but I believe it is licensed under the public domain. If this is not the case, please leave a comment below letting me know what the real license is.
This wordlist looks like a good starting point for many word based programming projects - and I don't want to loose this wordlist when Google Code shuts down for good.
You can find it here: enable1.txt
If you know if any other wordlists that would be useful for programmers (password cracking lists will NOT be considered), leave a comment with a link and a description below and I will look into hosting it here as well. Perhaps I can build up a collection of useful wordlists.