04-06-2016 - The most provocative algorithm ever made You'd think that playing music without skippping would be something users would demand from Spotify. Seriosuly, how hard can it be to get this right? An analog record player does this without missing a beat (pun belabored). However, judging by the sheer rage ciculating forums and comment sections online, it seems their users are orders of magnitude more upset over their shuffe algorithm. These are some of the comments in an ascending order of rage. "For the past couple of weeks I'll skip the same song repeated 1-2 in a string of about 10 songs, out of 4000+ from my starred list." "I've had the same song from my 1,100 track starred list play twice in my 35 minute commute." "Just did a quick test and my playlist of 48 tracks will only play 10 of the same songs over and over again and none of the other songs." "I've noticed this and am convinced that random isn't random. I imagine their random is actually a profit maximization model." "I listen to playlists with about 100 songs and with shuffle on, it will regularly repeat songs and only play a select number of songs instead of going through the whole playlist. This is with repeat off. A song will literally repeat after it has just played for goodness sake, something isn't right here.." "Are you fucking kidding me spotify? This whole article? How about another is played a second time! I've had premium over a year. You guys are ass." So what is going on? At first I chalked this up to the Gamblers Fallacy [1]. WIRED wrote an article explaining why true randomness is clustered, and not evenly distributed like most people intuit. This is the same reason the Brits mistakenly assumed that the Germans had an exceptionally good aim with their V-1 flying bombs during World War II [2]. With that said, I have a hard time seeing how the engineers at Spotify could mess this up. Shuffle is basically self-explanatory, especially if you've ever held a deck of cards. To shuffle a playlist you simply change the order of the songs randomly, just like you would when playing a card game. I rarely activate shuffle on my own playlists, which made it hard to confirm the alleged shortcomings above. Fortunately it is quite easy to interact with the user interface using AppleScript. #!/usr/bin/env bash function spotify { osascript -e "tell application \"Spotify\" to $*" } function play_all_the_songs { for i in {1..100} do spotify next track ARTIST=`spotify artist of current track` TRACK=`spotify name of current track` printf "%s - %s\n" "$ARTIST" "$TRACK" done } if [[ $1 = "continous" ]]; then spotify set shuffling to false elif [[ $1 = "shuffle" ]]; then spotify set shuffling to true fi play_all_the_songs The script above presumes that you have Spotify running and a playlist containg 100 unique songs opened. It plays through the entire playlist and writes the name of the artist and the song name of the current track to stdout. The first argument determines if shuffle should be activated or not. $ ./listen-through-playlist.sh continous Paolo Nutini - New Shoes Radiohead - Creep The Cars - Drive Kendrick Lamar - Backseat Freestyle [...] It is trivial to assert that the playlist in fact shuffles the music without playing any song more than once. Simply diff the output from a continous run against a shuffled run. They contain the same tracks. $ diff <(./listen.sh continous | sort) <(./listen.sh shuffle | sort) --report-identical-files Files /dev/fd/11 and /dev/fd/12 are identical The claim that a song is played more than once, effectively duplicating it in the queue, is false. In other words, Spotify is undeservingly on the receiving end of a shit storm caused by a failure of intution. [1] https://www.wired.com/2012/12/what-does-randomness-look-like/ [2] https://www.britannica.com/topic/Poisson-distribution 24-07-2015 - Red-Green-Refactor Last Christmas I decided to give TDD a try after being inspired from Gary Bernhardts screencasts at DestroyAllSoftware.com and reading Robert C. Martins book "Clean code". After overcoming the counterintuitive nature of writing tests before the actual code, I discovered that TDD improves my productivity and overall confidence in the code I write. In this post I try to define what I regard as a good test, why I like practicing TDD and why integration tests are not that useful. Most of the ideas in here are borrowed from Robert C. Martin, Gary Bernhardt, Martin Fowler and Kent Beck. # What to test? TDD is hard. 7 months in I still find it difficult to write valuable tests. This is actually the biggest challenge you'll run into when starting out: What tests should should you write? There is no definite answer, but I've found that testing code that I have a tendency to screw up is a good way to start. However, that's a pretty pretty vague and relative heurustic so let me demonstrate what I would describe as a worthless test: describe('Person', () => { describe('name', () => { it('should be possible to set', () => { var mike = new Person(); mike.name = 'Michael'; assert.equal(mike.name, 'Michael'); }); }); }); In order to screw up the logic that makes this test pass I would somehow have to mess up the setter-method of the name-field in the Person class. I find that unlikely and therefore consider this test as unnecessary. This becomes even clearer if write the code that makes that test pass. class Person { } That's it. The test would now pass. In other words: The sole existence of the Person class is enough to pass this test. Let's look at different example of a test I consider more valuable. describe('Person', () => { describe('name', () => { it('should be separated by whitespace if first and last name is set', () => { var mike = new Person(); mike.firstName = 'Michael'; mike.lastName = 'McMillan'; assert.equal(mike.name, 'Michael McMillan'); }); it('should only return first name if last name is not set', () => { var michelle = new Person(); michelle.firstName = 'Michelle'; assert.equal(michelle.name, 'Michelle'); }); it('should only return last name if first name is not set', () => { var mcLovin = new Person(); mcLovin.lastName = 'McLovin'; assert.equal(mcLovin.name, 'McLovin'); }); }); }); This code would pass that test [1]. class Person { get name () { return ((this.firstName || '') + ' ' + (this.lastName || '')).trim(); } } Although this may seem like a no-brainer, it's not something I want my code to mess up. I want confidence in that a persons name is properly formatted throughout my application. This test will ensure that. If anyone were to mess around with that code at least one of those tests would necesarrily fail and hopefully keep a bug from entering the code. # Monstrous tests Practicing TDD implies unit testing. Unit testing is hard to define and I don't want to be pissing on anyones bonfire [2], so I'll rather just piggyback off Gary Bernhardts definition: A unit test is when "Other classes cant break [the test]". In the previous example no other class can interfere with the result of our test because we are only testing the only relevant class: Person. For that reason we can claim that we are testing the logic of a Persons name in *isolation*. It is perfectly fine to write tests that are not unit tests, I just find them to be a lot less valuable, especially system tests, also known as integration tests. TDD provides you regression, design and easier refactoring. Integration tests on the other hand can only tell you if you have broken something (regression) somewhere in the stack. Not to mention they take a lot of time to run. Imagine you have an application exposing a REST API over HTTP. In fact, let's pretend it's the same application as the one containing the Person class above. Had we written an integration test instead of a unit test our test would probably look something like this. describe('Person', () => { describe('name', () => { it('should be separated by whitespace if first and last name is set', (done) => { request .post('http://localhost:5000/api/person') .send({ firstName: 'Michael', lastName: 'McMillan' }) .end(function(err, res){ var mike = res.request.toJSON(); assert.equal(mike.name, 'Michael McMillan'); done(); }); }); }); }); Although we are testing the same logic as before, we are dependent on a whole lot of classes and dependencies to not break us. More specifically: The web server/proxy (nginx/Apache), the router (Express), (perhaps) the database and the HTTP client. Additionally it takes a lot more time to complete this test. In fact, without mocking the network layer this is what would happen behind the scene when running it [3]: 1. Client/Server establishes a TCP socket (handshake) 2. Client POSTs the HTTP request 3. Server receives HTTP request 4. Server parses HTTP request and routes to correct controller 5. Controller constructs a new Person class with request parameters 6. Person class applies logic to return the name correctly 7. Sever responds to client with a JSON serialized Person object 8. Client parses response and asserts that the name is correct If one of the steps above fail we've been broken by code that essentially has nothing to do with the actual logic we're testing: A Persons first and last name should be separated with whitespace. However, this won't go unoticed: Our test will kindly inform us that it didn't pass, from there we need to dig through the entire stack in order to hunt down the bug that broke the test. It could be anywhere, it could even be located in code we haven't written ourselves. # There's a pattern here somewhere ... TDD forces you to focus on the application specific logic. If you were told to create a web application that had a simple HTML form that POST'ed a word to the backend and responded with its Scrabble score, where would you start? I think most developers would get so disorientated by the word "web application" that they'd start by installing a web framework like Ruby on Rails, Sails.js/Express or Laravel and start configuring routes and controllers in order to complete the task. But by practicing TDD (or Unit Testing for that matter) you would be forced to start with writing a simple test. Something along these lines is probably where you'd start: describe('Scrabble', () => { describe('score', () => { it('should be 0 for empty word', () => { var word = new ScrabbleWord(' '); assert.equal(word.score, 0); }); it('should be 20 for the word "quick"', () => { var word = new ScrabbleWord('quick'); assert.equal(word.score, 20); }); it('should be 20 for the word "quick" in capital letters', () => { var word = new ScrabbleWord('QUICK'); assert.equal(word.score, 20); }); it('should be 20 for the word "quick" reversed', () => { var word = new ScrabbleWord('KCIUQ'); assert.equal(word.score, 20); }); it('should be 7 for the word "slow"', () => { var word = new ScrabbleWord('slow'); assert.equal(word.score, 7); }); }); }); As you can see our test is completely agnostic as to whether this is a web application or not. This test has essentially helped us separate the business logic of calculating Scrabble scores from the I/O channel (HTTP). Imagine if the next task was to transform this Scrabble calculator into an IRC bot, a command line tool or even a desktop application with a GUI. We would only need to change the layer that interacts with the client - *not* the entire framework. My example may seem contrived due to the small amount of code that requires to complete the task of setting up a simple app calculating Scrabble scores, after all the algorithm computing the score more or less consists of a for-loop and a hashtable. But as with most software, complexity gets added with time. If this Scrabble calculator one day were to be converted into an actual Scrabble game you would only have to write more tests (with corresponding code of course) to stay decoupled. Footnotes: [1] I arrived at that code after iterating over red-green-refactor three times. I skipped that process in order to keep it short and concise. [2] DHH on TDD unit test definition: http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html [3] https://github.com/alex/what-happens-when#dns-lookup 25-04-2014 - Studjobb While looking for a summerjob online, I realized how tedious and uneffective the whole process was. Scouring 3-5 websites in hopes of finding someone willing to hire a freshman is not easy. Mostly due to the lack of filtering. Jason Fried (founder and CEO at Basecamp) said something that stuck with me for a while. "[...] It's important to solve your own problems, and to recognize that your problem is not unique." I decided to scratch my own itch to see if someone else shared my annoyance. Thanks to the great prototyping framework Bootstrap, it didn't take long before I had Studjobb.no up and running. My goal was to make a simple and lazy way of receiving job postings for students. After some back-and-forth I ended up with a weekly mailing list and a simple table featuring the latest jobs. I also added the ability for companies to advertise their positions. So far I've only spread the word about the site through a couple of internal mailing lists at NTNU. The service is completely automated, the only thing that remains is getting the word out. 01-12-2013 - Referencing sucks Paul Graham's essays are always entertaining to read. My favourite essay, which I also try to keep in the back of my head whenever a "brilliant" idea comes along, is How to Get Startup Ideas. "The way to get startup ideas is not to try to think of startup ideas. It's to look for problems, preferably problems you have yourself." One week ago I decided to solve a problem which has been an annoyance for a long time: Referencing. Throughout high-school, being able to reference books, articles and websites is an important part of the curriculum. Failing to do so will almost certainly negativly affect your grade. In combination with Bootstrap, Angular, a simple crawler, a MySQL-backend glued together with a REST API and some good ol' elbow grease - LittList was born. LittList lets you search for books by author, title, ISBN or content, in a huge database consisting of metadata for books. The student then proceeds by adding the references that were used. Lastly the student clicks Generate and a properly formatted bibliography is returned. I launched the site on the evening of the 24th of November by posting the link to a couple of private Facebook Groups. The response was overwhelmingly positive! I quickly received a lot of feedback regarding the technical implementation and the general userinterface. It was extremely fun, not to mention addictive, to monitor the traffic in realtime with Piwik, and getting appreciative e-mails. In the course of 5 days the site had received about 5.2 thousand unique visitors and over 4.8 thousand bibliographies had been generated. It did not take long before other websites started noticing the post on Facebook, which was rapidly gaining traction (in the form of likes and comments). As a result, StudentTorget.no published an article which led to even more traffic. Due to a relativly poor and inconsiderate programmed crawler, the CPU of the server LittList was hosted on started to choke. Luckily, I managed to resolve the issue before crashing the entire box. Even though only one week has passed, it has been very rewarding to hear that LittList actually helps students who are fed up with spending time formatting bibliographies. I guess writing the assignment/paper in the first place is boring enough. 27-11-2013 - Trondheim It would be an understatement to say that I'm excited for what my time in Trondheim will bring. Being able to study Computer Science has been a long-term goal, thus being accepted at NTNU is undoubtely a huge achievement for me. One of the first things I noticed in Trondheim was the great diversety within extracurricular activites you could take part in. There is literally something for everyone. I was by coincidence introduced to two programs in late September: AppLab and SPARK. Both of these programs aims to guide and mentor students with startup-ideas towards a minimum viable product. I chose to participate in both of them, as AppLab mainly focuses on building an app for mobile platforms, and SPARK provides tight follow-ups and personal advisory from an NSE-student [1]. In retrospect I can safely say that I feel very much at home in Trondheim. Being given the opportunity to work on an idea - whether it's good or bad - is extremely rewarding and fun! It's a shame that programs which try to encourage innovation only seem to be available at university. I would have loved to attend something similar in high-school. I'm really looking forward to the following years.