04-06-2016 - The most provocative algorithm ever made
You'd think that playing music without skippping would be something
users would demand from Spotify. Seriosuly, how hard can it be to get
this right? An analog record player does this without missing a beat
(pun belabored).
However, judging by the sheer rage ciculating forums and comment
sections online, it seems their users are orders of magnitude more
upset over their shuffe algorithm.
These are some of the comments in an ascending order of rage.
"For the past couple of weeks I'll skip the same song repeated 1-2
in a string of about 10 songs, out of 4000+ from my starred list."
"I've had the same song from my 1,100 track starred list play
twice in my 35 minute commute."
"Just did a quick test and my playlist of 48 tracks will only play
10 of the same songs over and over again and none of the other
songs."
"I've noticed this and am convinced that random isn't random. I
imagine their random is actually a profit maximization model."
"I listen to playlists with about 100 songs and with shuffle on,
it will regularly repeat songs and only play a select number of
songs instead of going through the whole playlist. This is with
repeat off. A song will literally repeat after it has just played
for goodness sake, something isn't right here.."
"Are you fucking kidding me spotify? This whole article? How about
another is played a second time! I've had premium over a year. You
guys are ass."
So what is going on? At first I chalked this up to the Gamblers
Fallacy [1]. WIRED wrote an article explaining why true
randomness is clustered, and not evenly distributed like most people
intuit. This is the same reason the Brits mistakenly assumed
that the Germans had an exceptionally good aim with their V-1 flying
bombs during World War II [2].
With that said, I have a hard time seeing how the engineers at Spotify
could mess this up. Shuffle is basically self-explanatory, especially
if you've ever held a deck of cards. To shuffle a playlist you simply
change the order of the songs randomly, just like you would when playing
a card game.
I rarely activate shuffle on my own playlists, which made it hard to
confirm the alleged shortcomings above. Fortunately it is quite easy
to interact with the user interface using AppleScript.
#!/usr/bin/env bash
function spotify {
osascript -e "tell application \"Spotify\" to $*"
}
function play_all_the_songs {
for i in {1..100}
do
spotify next track
ARTIST=`spotify artist of current track`
TRACK=`spotify name of current track`
printf "%s - %s\n" "$ARTIST" "$TRACK"
done
}
if [[ $1 = "continous" ]]; then
spotify set shuffling to false
elif [[ $1 = "shuffle" ]]; then
spotify set shuffling to true
fi
play_all_the_songs
The script above presumes that you have Spotify running and a playlist
containg 100 unique songs opened. It plays through the entire playlist
and writes the name of the artist and the song name of the current
track to stdout. The first argument determines if shuffle should be
activated or not.
$ ./listen-through-playlist.sh continous
Paolo Nutini - New Shoes
Radiohead - Creep
The Cars - Drive
Kendrick Lamar - Backseat Freestyle
[...]
It is trivial to assert that the playlist in fact shuffles the music
without playing any song more than once. Simply diff the output from a
continous run against a shuffled run. They contain the same tracks.
$ diff <(./listen.sh continous | sort) <(./listen.sh shuffle | sort) --report-identical-files
Files /dev/fd/11 and /dev/fd/12 are identical
The claim that a song is played more than once, effectively
duplicating it in the queue, is false. In other words, Spotify
is undeservingly on the receiving end of a shit storm caused
by a failure of intution.
- M
[1] https://www.wired.com/2012/12/what-does-randomness-look-like/
[2] https://www.britannica.com/topic/Poisson-distribution
24-07-2015 - Red-Green-Refactor
Last Christmas I decided to give TDD a try after being inspired from
Gary Bernhardts screencasts at DestroyAllSoftware.com and reading
Robert C. Martins book "Clean code". After overcoming the
counterintuitive nature of writing tests before the actual code, I
discovered that TDD improves my productivity and overall confidence in
the code I write.
In this post I try to define what I regard as a good test, why I like
practicing TDD and why integration tests are not that useful. Most of
the ideas in here are borrowed from Robert C. Martin, Gary Bernhardt,
Martin Fowler and Kent Beck.
# What to test?
TDD is hard. 7 months in I still find it difficult to write valuable
tests. This is actually the biggest challenge you'll run into when
starting out: What tests should should you write? There is no definite
answer, but I've found that testing code that I have a tendency to
screw up is a good way to start. However, that's a pretty pretty
vague and relative heurustic so let me demonstrate what I would
describe as a worthless test:
describe('Person', () => {
describe('name', () => {
it('should be possible to set', () => {
var mike = new Person();
mike.name = 'Michael';
assert.equal(mike.name, 'Michael');
});
});
});
In order to screw up the logic that makes this test pass I would
somehow have to mess up the setter-method of the name-field in the
Person class. I find that unlikely and therefore consider this test as
unnecessary. This becomes even clearer if write the code that makes
that test pass.
class Person {
}
That's it. The test would now pass. In other words: The sole existence
of the Person class is enough to pass this test. Let's look at
different example of a test I consider more valuable.
describe('Person', () => {
describe('name', () => {
it('should be separated by whitespace if first and last name is set', () => {
var mike = new Person();
mike.firstName = 'Michael';
mike.lastName = 'McMillan';
assert.equal(mike.name, 'Michael McMillan');
});
it('should only return first name if last name is not set', () => {
var michelle = new Person();
michelle.firstName = 'Michelle';
assert.equal(michelle.name, 'Michelle');
});
it('should only return last name if first name is not set', () => {
var mcLovin = new Person();
mcLovin.lastName = 'McLovin';
assert.equal(mcLovin.name, 'McLovin');
});
});
});
This code would pass that test [1].
class Person {
get name () {
return ((this.firstName || '') + ' ' + (this.lastName || '')).trim();
}
}
Although this may seem like a no-brainer, it's not something I want my
code to mess up. I want confidence in that a persons name is properly
formatted throughout my application. This test will ensure that. If
anyone were to mess around with that code at least one of those tests
would necesarrily fail and hopefully keep a bug from entering the code.
# Monstrous tests
Practicing TDD implies unit testing. Unit testing is hard to define
and I don't want to be pissing on anyones bonfire [2], so I'll rather
just piggyback off Gary Bernhardts definition: A unit test is when
"Other classes cant break [the test]". In the previous example no
other class can interfere with the result of our test because we are
only testing the only relevant class: Person. For that reason we can
claim that we are testing the logic of a Persons name in *isolation*.
It is perfectly fine to write tests that are not unit tests, I just
find them to be a lot less valuable, especially system tests, also
known as integration tests. TDD provides you regression, design and
easier refactoring. Integration tests on the other hand can only tell
you if you have broken something (regression) somewhere in the stack.
Not to mention they take a lot of time to run.
Imagine you have an application exposing a REST API over HTTP. In
fact, let's pretend it's the same application as the one containing
the Person class above. Had we written an integration test instead of
a unit test our test would probably look something like this.
describe('Person', () => {
describe('name', () => {
it('should be separated by whitespace if first and last name is set', (done) => {
request
.post('http://localhost:5000/api/person')
.send({ firstName: 'Michael', lastName: 'McMillan' })
.end(function(err, res){
var mike = res.request.toJSON();
assert.equal(mike.name, 'Michael McMillan');
done();
});
});
});
});
Although we are testing the same logic as before, we are dependent on
a whole lot of classes and dependencies to not break us. More
specifically: The web server/proxy (nginx/Apache), the router
(Express), (perhaps) the database and the HTTP client. Additionally it
takes a lot more time to complete this test. In fact, without mocking
the network layer this is what would happen behind the scene when
running it [3]:
1. Client/Server establishes a TCP socket (handshake)
2. Client POSTs the HTTP request
3. Server receives HTTP request
4. Server parses HTTP request and routes to correct controller
5. Controller constructs a new Person class with request parameters
6. Person class applies logic to return the name correctly
7. Sever responds to client with a JSON serialized Person object
8. Client parses response and asserts that the name is correct
If one of the steps above fail we've been broken by code that
essentially has nothing to do with the actual logic we're testing: A
Persons first and last name should be separated with whitespace.
However, this won't go unoticed: Our test will kindly inform us that
it didn't pass, from there we need to dig through the entire stack in
order to hunt down the bug that broke the test. It could be anywhere,
it could even be located in code we haven't written ourselves.
# There's a pattern here somewhere ...
TDD forces you to focus on the application specific logic. If you were
told to create a web application that had a simple HTML form that
POST'ed a word to the backend and responded with its Scrabble score,
where would you start?
I think most developers would get so disorientated by the word "web
application" that they'd start by installing a web framework like Ruby
on Rails, Sails.js/Express or Laravel and start configuring routes and
controllers in order to complete the task.
But by practicing TDD (or Unit Testing for that matter) you would be
forced to start with writing a simple test. Something along these
lines is probably where you'd start:
describe('Scrabble', () => {
describe('score', () => {
it('should be 0 for empty word', () => {
var word = new ScrabbleWord(' ');
assert.equal(word.score, 0);
});
it('should be 20 for the word "quick"', () => {
var word = new ScrabbleWord('quick');
assert.equal(word.score, 20);
});
it('should be 20 for the word "quick" in capital letters', () => {
var word = new ScrabbleWord('QUICK');
assert.equal(word.score, 20);
});
it('should be 20 for the word "quick" reversed', () => {
var word = new ScrabbleWord('KCIUQ');
assert.equal(word.score, 20);
});
it('should be 7 for the word "slow"', () => {
var word = new ScrabbleWord('slow');
assert.equal(word.score, 7);
});
});
});
As you can see our test is completely agnostic as to whether this is a
web application or not. This test has essentially helped us separate
the business logic of calculating Scrabble scores from the I/O channel
(HTTP). Imagine if the next task was to transform this Scrabble
calculator into an IRC bot, a command line tool or even a desktop
application with a GUI. We would only need to change the layer that
interacts with the client - *not* the entire framework.
My example may seem contrived due to the small amount of code that
requires to complete the task of setting up a simple app calculating
Scrabble scores, after all the algorithm computing the score more or
less consists of a for-loop and a hashtable. But as with most
software, complexity gets added with time. If this Scrabble calculator
one day were to be converted into an actual Scrabble game you would
only have to write more tests (with corresponding code of course) to
stay decoupled.
Footnotes:
[1] I arrived at that code after iterating over red-green-refactor
three times. I skipped that process in order to keep it short and
concise.
[2] DHH on TDD unit test definition:
http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html
[3] https://github.com/alex/what-happens-when#dns-lookup
- M
25-04-2014 - Studjobb
While looking for a summerjob online, I realized how tedious and
uneffective the whole process was. Scouring 3-5 websites in hopes of
finding someone willing to hire a freshman is not easy. Mostly due to
the lack of filtering.
Jason Fried (founder and CEO at Basecamp) said something that stuck with
me for a while.
"[...] It's important to solve your own problems, and to recognize
that your problem is not unique." I decided to scratch my own itch to
see if someone else shared my annoyance. Thanks to the great
prototyping framework Bootstrap, it didn't take long before I had
Studjobb.no up and running.
My goal was to make a simple and lazy way of receiving job postings
for students. After some back-and-forth I ended up with a weekly
mailing list and a simple table featuring the latest jobs. I also
added the ability for companies to advertise their positions.
So far I've only spread the word about the site through a couple of
internal mailing lists at NTNU. The service is completely automated,
the only thing that remains is getting the word out.
- M
01-12-2013 - Referencing sucks
Paul Graham's essays are always entertaining to read. My favourite
essay, which I also try to keep in the back of my head whenever a
"brilliant" idea comes along, is How to Get Startup Ideas.
"The way to get startup ideas is not to try to think of startup ideas.
It's to look for problems, preferably problems you have yourself." One
week ago I decided to solve a problem which has been an annoyance for
a long time: Referencing. Throughout high-school, being able to
reference books, articles and websites is an important part of the
curriculum. Failing to do so will almost certainly negativly affect
your grade.
In combination with Bootstrap, Angular, a simple crawler, a
MySQL-backend glued together with a REST API and some good ol' elbow
grease - LittList was born. LittList lets you search for books by
author, title, ISBN or content, in a huge database consisting of
metadata for books. The student then proceeds by adding the references
that were used. Lastly the student clicks Generate and a properly
formatted bibliography is returned.
I launched the site on the evening of the 24th of November by posting
the link to a couple of private Facebook Groups. The response was
overwhelmingly positive! I quickly received a lot of feedback
regarding the technical implementation and the general userinterface.
It was extremely fun, not to mention addictive, to monitor the traffic
in realtime with Piwik, and getting appreciative e-mails. In the
course of 5 days the site had received about 5.2 thousand unique
visitors and over 4.8 thousand bibliographies had been generated.
It did not take long before other websites started noticing the post
on Facebook, which was rapidly gaining traction (in the form of likes
and comments). As a result, StudentTorget.no published an article
which led to even more traffic. Due to a relativly poor and
inconsiderate programmed crawler, the CPU of the server LittList was
hosted on started to choke. Luckily, I managed to resolve the issue
before crashing the entire box.
Even though only one week has passed, it has been very rewarding to
hear that LittList actually helps students who are fed up with
spending time formatting bibliographies. I guess writing the
assignment/paper in the first place is boring enough.
- M
27-11-2013 - Trondheim
It would be an understatement to say that I'm excited for what my time
in Trondheim will bring. Being able to study Computer Science has been
a long-term goal, thus being accepted at NTNU is undoubtely a huge
achievement for me.
One of the first things I noticed in Trondheim was the great diversety
within extracurricular activites you could take part in. There is
literally something for everyone. I was by coincidence introduced to
two programs in late September: AppLab and SPARK.
Both of these programs aims to guide and mentor students with
startup-ideas towards a minimum viable product. I chose to participate
in both of them, as AppLab mainly focuses on building an app for
mobile platforms, and SPARK provides tight follow-ups and personal
advisory from an NSE-student [1].
In retrospect I can safely say that I feel very much at home in
Trondheim. Being given the opportunity to work on an idea - whether
it's good or bad - is extremely rewarding and fun! It's a shame that
programs which try to encourage innovation only seem to be available
at university. I would have loved to attend something similar in
high-school.
I'm really looking forward to the following years.
- M