CDNperf - CDN Performance in Numbers

The big day has come. It's time to reveal CDNperf. The service has been designed to provide a quick glance at various JavaScript CDN providers. Before this it has been very difficult to get an idea of their performance. Not anymore.

Besides providing this sort of service we aim to extend CDNperf to contain web performance related content. It is one of those topics that has been scattered around the web. Perhaps we can do something about that.

We have had the site in a beta for a while now. I am not saying it's entirely complete yet but it should be okay enough for casual usage or a glance at least. In case you are interested in the technology used, have a look at the GitHub repository.

This is a project that simply would not exist without our partners. It started as a collaboration between us JSter guys and Dmitriy Akulov of jsDelivr. We simply wanted to create something together and this lead to an idea of a CDN performance site.

As measuring performance can be difficult and requires global infrastructure, we contacted Pingdom. Thankfully they agreed to sponsor our efforts. To make it all work we also needed a host. BlueVM answered to the call. Big thank you!

Even though CDNperf might not be technically the most complex project, it proved to me that you can get things done with a bit of collaboration. It would have been next to impossible for me or Dmitriy alone to build a service such as this. With a bit of persuasion and hard work we managed to do it.

I hope you enjoy using the service. In case you happen to have any specific ideas on how to make it better, do let us know at the comment section.

Thoughts on Devaamo Summit 2013 and treweb

Guess what, it was Devaamo Summit time of year again! This time it was a double event as they organized treweb, a web focused unconference while at it. I visited the conference the first time last year and it was great fun. For some reason there were less people around this year but it was still quite nice. It's all about the quality as you know.

SurviveJS - Redux

I posted a talk proposal to the Summit at the end of April. That was direct continuation to my previous talk at AgileJkl. The idea was to prepare something shorter and more focused. As I didn't hear from the guys after that, I thought they ditched the idea. You can bet I was surprised when I discovered my name on the official program yesterday.

As I did not want to disappoint the organizers, I decided to wing it and use my AgileJkl slides in my 30 minute slot. Overall the presentation felt nice (from this point of view) and I think I might have gotten a few points through. I made the whole more practical by showcasing actual implementations and tools. It felt like a good idea to get the audience involved a little bit.

Somehow the stage was easier this time as the audience was on the same floor level as you and it was very easy to question them. Maybe it wasn't that bad a thing not to prepare too much after all. Anyway, it was yet another good learning experience. I think caffeine added some extra buzz to the whole.

In case you are interested in the associated material, check out the companion site of the presentation at Survive JS. It delves way deeper in the topics discussed and should provide some food for thought for the technically minded.

Thoughts on Presentations

Overall the presentations I saw were at least adequate quality. Yes, there were some perhaps more boring ones than others but that's just fine. At least the short slots more or less force you to get to the point eventually. I know mine would have been more effective had I trimmed it. If I get to present the ideas again, I just might do that.

Jolla - Keynote

Marc Dillon, the former CEO of Jolla, gave the keynote of the conference. As you might already know, Jolla is Finnish for dinghy and is a company that spun off Nokia. Quite apt naming, eh? They're building on the legacy of MeeGo and aim to shake the mobile world in their own way. Who says the lightning doesn't strike twice in the same spot?

Android and Fragmentation

Marc made some interesting points about how Android is become more similar (less variance between vendors) and the fact how walls are erected between Google and the developers. I am actually not sure whether making the Android ecosystem more homogenous on the whole is a bad thing. At least in my mind the current state of fragmentation that Google needs to take seriously as it impedes the value of the ecosystem.

Android is a very difficult target because of this. There are simply so many possible configurations around it is difficult to test everything. Contrast this with something simple like Apple's iOS and you understand why it is such an issue.

The Other Half

I was initially somewhat skeptical about the concept of "other half" introduced by Jolla. Marc alleviated my concerns a little bit. Now I understand that it's more than just a gimmick such as Nokia's Xpress-On (N79) which could manipulate the system theme based on the cover inserted. Rather it's a hardware extension point for the whole phone.

Imagine if you could attach some new sensors or perhaps a QWERTY keyboard to your phone. The concept seems to allow this. I know these sort of things are still very much vaporware. Still, you have to admit there's some idea. We shall see whether it's enough.

Sailfish OS

Marc didn't go into the specifics of Sailfish, the operating system they are developing. It was glanced at a high level. Basically he stated it is more open than Android and given that the business starts to grow, they'll open it more. He did make the point that it is essential for a company to retain at least part of the secret source to itself even if it dealt with open source.

I think this might be something that's more critical for companies such as Jolla. They cannot afford to allow some other company to replicate their whole business. My guess is that the secret bit is there just to counter this scenario.

The Future of Jolla

It seems like Jolla tries to position itself somewhere between hardware and software. If I understood correctly, they want to be more like a community shepherd and enable growth on top of their system. Eventually this should lead to them working as a licensor so that bigger companies, such as Samsung, could build on top of their technology.

It remains to be seen whether they can reach some critical mass needed for this type of strategy to work. Especially the Samsung case seems intriguing. Why would Samsung license Sailfish rather than to focus on some of their own initiatives? What value could Sailfish possibly provide Samsung could not otherwise reach? I hope the Jolla guys have a good idea.

Enabling Open Data Communities

Jarkko Moilanen of Avoindata.net discussed the topic of how to enable open data related communities to emerge. We're still pretty new to this concept of open data here in Finland and are just starting to really open up our data reserves for the public. Jarkko discussed some of the challenges involved.

The Need for Vocabulary

Interestingly one of the main challenges seems to be finding the right Finnish vocabulary to discuss the concepts related to the topic. This sort of work is required so that things may be communicated effectively in the right places for progress to happen.

The Value of Open Data

Open data provides clear value to commerce. Unfortunately opening data is not that simple. Besides money it takes certain amount of political will and resolve. The site, Avoindata.net, developed by Jarkko aims to ease this process. It seems to have gained some traction already.

Monthly Meetups

Besides maintaining that Stack Overflowish site, they've arranged monthly meetups for around half a year at the time of writing. The meetups have been a great success. Interestingly it has lead more data to become available. Deadlines motivate even government people it seems.

The meetups foster interchange and enable new ideas to emerge. The format seems quite simple. There are just a couple of short presentations and some working together. Unfortunately it seems the time available isn't enough to reach concrete enough results. Currently they are thinking of arranging a hackathon type of event to remedy this.

Other Thoughts

It seems there's some seriously good buzz on the area of open data around Tampere. It seems they understand the value it provides for the local business and are willing to work for it. I can only hope the local powers that be at Jyväskylä understand the same thing very soon as well. At least there are some signs of that already.

HSL Navigator

Tuukka Hastrup gave a talk on HSL Navigator, a web based concept for bus passengers. We have navigators for cars already but what if we had one for these guys as well?

If I've understood correctly, HSL takes care of the public transportation around the metropolitan area (Helsinki, whatnot). They've made some great progress in opening data. In 2009 around 30 apps had been built on top of it. In 2012 the figure was 670. That's a massive increase and shows the power of developers.

Tuukka showed a visualization I found very interesting, a journey time map. You simply provide your current location and it shows on the map how long it takes to take from that spot to nearby areas. There's some sophisticated math behind that.

I'll get back to this topic at my summary of the last talk as it was related to this one.

Traffic API Developer Sandbox

Tero Piirainen of ITS Factory discussed what they've been doing with traffic data over at Tampere region. One of their primary objective seems to be to generate business by making traffic data available.

There are many sources. These include road traffic, geodata and public transport to mention some. Currently the data that is available is often in non-standard, proprietary format. They aim to change this and want to use open, standard formats instead.

As I said earlier, they seem to have some good buzz going on at Tampere and this presentation confirms it. They've acknowledged the value and seem to be willing to work for it.

Firefox OS

Thomas Rücker of Tieto gave a talk on Firefox OS. To be more accurate it was more of a talk of a talks given multiple slidesets were used. All I know about Firefox OS before this talk was that it was web based. The talk clarified some details.

The platform itself is very affordable, somewhere around 100 euros. Besides buying a new phone, you might get it to run on your current Android although apparently there may be some issues with resolutions. But there are supposedly hacks for that.

The nice thing, depending on your viewpoint, of the platform is that you get to do everything in JavaScript. There are hardware level APIs available natively. As you might not want to give everyone at the web access over your phone, there is some kind of an elevation based security model included as well. You can decide not to allow that nice app to send SMS messages unless you really want to.

To be honest, out of Sailfish and Firefox OS the latter one seems more promising to me at the moment. At least they've gotten out of the vaporware stage and have something concrete to show. I would not mind if it became a great success. Even as a minor success it might force some of the bigger players to rethink their approach and make web a first class citizen on their platforms.

The next sections will be shorter as they were part of treweb and more technical in nature. I've tried to pick up the juiciest bits rather than boring you with the details.

Typography ABC

Antti Mattila talked about the basics of typography. I won't go into details as you can pick up the basics at a resource such as Hack Design. I will mention a couple of core points:

• Learn to pair typefaces
• Don't use too small typeface. Consider poor eyesight and all that
• Most of Google Fonts aren't that good. Use Beautiful Web Type to find the good ones
• Font Squirrel has a nice set of free fonts available
• Of commercial ones Typekit and Fontdeck should be nice
• Fonts are expensive for a reason. You get what you pay for

CSS Flexbox

Janne Kallunki discussed CSS flexbox. It's one of those features that would make the daily drudgery with CSS so much nicer. It provides some well needed functionality and turns some difficult things into almost intuitive. Rather than having to fight with floats and display options your declarations become more natural.

The biggest issue with the feature seems to be poor support. There are multiple versions of the specification. This has lead to some major fragmentation between browsers. And then we have IE. I won't go into that.

It looks like it's still too early to use flexbox in production. The feature shows a great deal of promise. I think it will take a couple of years till we see it in the mainstream. Even then it will be likely with some kludges.

This, That and the Self

Antti Järvinen talked about this. I mean that. Okay, bad pun. Anyway, the talk was interesting and highlighted various corner cases. Personally I like to avoid this and too object oriented code these days but perhaps that's just me. You still have to be aware of the basic binding rules.

The biggest gotchas seem to be related to closures as they don't work as intuitively as you might expect. I remember tripping on this when I got started with JavaScript. Remember, if you use a closure and want to use this, you will likely want to use that instead. Oh, and be careful with jQuery and this. It does some strange things to it.

Web Frameworks on Cloud Platforms - Pitfalls and Improvements

Jukka Tupamäki's presentation was probably the most academic one in the track. I won't go into much detail here. I expect you know cloud is the shiz, it's elastic and everything. Even though cloud is the new hip thing, frameworks haven't quite kept up with the progress. As a result there is all sorts of nastiness around the developers have to deal with.

When you start to scale and share data and logic between servers, traditional approaches fall apart. Your cronjobs might start running multiple times and wreaking havoc on your database. Data might get stored on the wrong server. You get the idea.

This leads to macro scale changes. You should take cloud and scaleability in count on your application design. If the architecture doesn't scale to multiple servers, you will be in big trouble at some point.

I think that in practice this means that you should aim to encapsulate your service into smaller compartments with well defined interfaces. This is perhaps starting to sound a bit like SOA could be a good idea.

Offline Transit Routing in JavaScript

Juha Järvi of Busfaster continued on where Tuukka left. Offline usage is particularly important for user groups such as tourists. Nobody wants to store a terabyte of map data on their phone, though, so how to get something that's useful enough? That's where various compression techniques come in.

First of all the whole map has been sliced to smaller areas. Within each area nearby points may be merged without sacrificing too much accuracy. Dead ends may be eliminated. There are these sort of small things which you can do to drastically remove data while still retaining enough so that valid results may be attained.

Even timetables may be compressed as they contain regular data. You simply need to store some sort of an offset rather than each time. Better yet the data doesn't have to be exactly correct. This provides room for further optimizations as you afford to let the data to be a minute or two off for instance.

On interesting scenario he mentioned is that the data could be served on some cloud service the user has access to. Dropbox could be a good alternative for instance. If they can make the workflow simple enough, I guess even the less nerdy of us would find this useful.

The technology stack they use was quite conventional. Basically just Closure Compiler, Node.js, PostGIS, Emscripten and asm.js (Emscripten target). I thought everyone used Uglify these days so Closure Compiler was bit of a surprise. I guess it must have its benefits especially in a case such as this.

Conclusion

It was very nice to visit Devaamo Summit again. treweb was just a nice bonus. The venue is just great for something like this. I hope they'll have more resources available next year so they can make the event even more awesome. It's just a matter of pulling the right strings.

I had actually prepared some bonus material for treweb that would have included some live coding. Perhaps it's better to save that for some another time. :)

Node.js - a Collection of Server Configuration Patterns

There are many ways to set up Node.js server configuration. Rather than writing a blog post about the topic and having to maintain that I started a repository that goes through a couple of patterns I have used. During its development I found myself writing yet another module, namely parse-env.

parse-env allows you to support environment variables based on your current configuration scheme as long as it is based on an Object structure. It simply constructs environment variable names using some simple logic and then checks whether they exist or not. This allows you to rely on convention rather than having to define each of those by hand.

This sort of approach is especially valuable in environments, such as Heroku, where you have a limited amount of control over the server. In case of Heroku you are pretty much forced to deal with environment variable based configuration as nobody in their right mind would commit configuration to the repository.

Let me know if I'm missing some vital configuration pattern and I'll look into adding it. Even better, poke me with a pull request and I'll be really happy. :)

Linkdump 14 - Business, Startups, Software Development...

To save some effort and make the dump more useful, I'll compile one now. I did one a month ago and have accumulated some more or less interesting content since. Enjoy the findings below!

Business

• Business Buzzword Bingo and Why You Shouldn't Play by Chuck Blakeman. Big words hide meaning and sound cheesy.
• Embrace Self-Disruption Using the Business Model Canvas
• Innovation Lessons from the Rise of Tesla Motors
• Deleted my portfolio, made $30k in my first six months…
• Self-Organizing Organizations (For Real)
• Survivorship Bias - The dead don't tell stories
• Organize for Complexity, part I+II. How to make work work again - Superb slides on the topic of complexity and organizations

Startups

• The Groundhog Day Effect and the curse of the flat line
• 3 Keys for Innovation: Why Lean Startup Isn’t Enough - Putting Lean Startup in context
• The Minimum Lovable Product - Delighting your clients is a good starting point
• The Money Making Techniques every startup should know - Awesome slides
• Stages of the Startup Lifecycle
• Search versus Execute
• Memo to this year’s YC class: It’s damn hard to build an enterprise company
• Who's Your Ideal Customer?
• Confessions of a lean startup: how I got my first customers without having a product
• Problem, Revenue, Channel: The 3 Critical Startup Questions
• Letter To A Young Programmer Considering A Startup
• Why You Should Start Marketing Before Writing Any Single Line Of Code

Personal Development

• Make Your Own Bubble in 10 Easy Steps - Isolate yourself from the suck
• You Can't Do Everything but you can do anything
• The Buffett Formula — How To Get Smarter

Software Development

• Automotive Metaphors for Lean and Agile Software Development
• The Wetware Crisis: TEPES - A model for hiring software people
• Locks, Actors, And STM In Pictures
• How to Safely Store a Password
• The Open Source Report Card - Generated based on your GitHub profile
• Programmer Interrupted
• Robot senses when you need a beer and will pour one for you - The future is here

Web Development

• Bringing functional to the frontend: Clojure + ClojureScript for the web
• Nibbler - Test your website!
• CSS Architecture - Avoid the jungle
• Sublime Text Resources for Front-end Developers
• Better web typography in a few simple steps
• This Is Responsive - Not Spartaaa!
• Slickmap CSS
• Best Practices for Designing a Pragmatic RESTful API

Watercolor - a Challenging Yet Rewarding Medium

I have a dark secret. I actually like to draw and paint. Those are not very programmery things to do. But there, I said it. You have to find your fun somewhere. :)

Painting is something I picked up just recently. Before that I've spent some time doodling every once in a while. I've spend some time on studying constructive anatomy (Bridgman) and have some idea of how to compose pictures. I suppose this is something that might help in web design.

During the last Winter/Spring I enrolled to a painting course. There was some charcoal work involved. Charcoal is interesting as it's a medium that allows you to both draw and paint. It is a very flexible yet demanding medium. And a very affordable one. Anyhow, it's good preparation for real painting.

The last time I touched watercolors before that was on the last millennia. I recall the results were not that great but I think it might have had something to do with the quality of the equipment. You don't need that much equipment but be sure not to skimp there too much. You can get a decent kit for around 50 dollars/euros with some starting papers and all.

Watercolor as a Medium

My toolkit

Watercolor is a challenging yet affordable medium. You can get started with two good brushes (small and big), a set of colors (a cheap Yarka one will do), a sponge (borrow a makeup sponge) and a water container (you can cut a milk carton).

Brushes come in many flavors. It's a good idea to spend a bit on quality to make the process less frustrating. That biggest one on the picture is actually a cheap but interesting brush known as hake. It's a brush popularized by Ron Ranson. I've yet to master it.

Besides these basic tools you'll need some paper and tape. Be sure to pick heavy enough paper (at least 180 gsm). That's another way to avoid frustration as it allows you wipe some of the color off using a sponge and overall allows you to be rougher while working. There are also techniques such as "wet on wet" that may be implemented on heavier papers.

If you want to end up with an even end result, you might want to consider using a specific kind of watercolor tape rather than regular one. It takes some extra effort to set up but it's worth it especially if you wish to frame your painting.

Watercolor + ??? = Awesomeness

Pencil study (~5 mins)

What makes watercolor interesting as a medium is the fact that you can combine it with some others. You could for instance create an underdrawing using pencils or waterproof markers. There are also specific kind of pencils which dissolve when touched with water.

You can also work on the painting while it's still wet and draw on it. Stabbing is also allowed (knife is useful!) and may lead to interesting effects. Or you could sprinkle some salt on it. There are no limits.

Painting Process

Before I start to paint, I usually prepare a preliminary drawing in which I examine basic shapes and their relations. Somehow it makes it easier to focus on the difficult parts, like color, while painting. In order to make it easier to deal with color, I like to paint a small color study on a card.

Color study

It's a good idea to paint a few paintings from the same subject in a row. You can even work on them simultaneously while waiting some parts to dry. Watercolors are not for the impatient.

If you manage to botch some, it's alright. Sometimes accidents can actually be a good thing and add a great deal of interest to your painting. And if it's really horrid, well, that's why rubbish cans were invented. But before scrapping the work, consider painting on the other side first. :)

The painting you can see below was from a series of four I painted within an hour or so. In this case I was in too much of a hurry to spend time on underdrawings. Sometimes it's just better to do than to think too much.

I recall spending around 15 minutes on this piece using mainly my big, sharp-edged brush. I like particularly how the bottle turned out. Interestingly each painting had a character of its own even though the subject was the same.

Framed result

Resources

I know reading this post didn't make you an instant master of watercolors. But I hope it at least provided some inspiration. I have a great deal to learn myself. As you learn one thing adequately, you'll find challenges elsewhere.

WetCanvas, a popular art forum, has a nice thread that might be worth looking into. You'll find some beginner and even advanced resources there. The following blogs might provide some ideas as well although their scope is way wider:

• Gurney Journey - In addition his books are excellent and well worth looking into.
• Freshdesigner - Besides posts he has some very nice material available at YouTube.
• Illustration Art - Just awesome examples from art history.
• Nathan Fowkes Art - His charcoal technique is just stunning. Take a good look at his demos.
• Stapleton Kearns - Tons of valuable information especially on the topic of painting.

How to Write a Simple Templating Engine in Python?

I have covered the concept of templating earlier on this blog. To recap the idea is simple. You have a template which contains "slots". It is possible to inject content into these "slots". You end up with a string that contains that new content injected to your template. Simple as that.

There are many applications for this basic technique ranging from web development to document generation.

Existing Templating Solutions

Python comes with a few templating mechanisms already. I've listed a few common ones below:

• '%d bottles of beer on the wall' % 50 # the old way
• '{amount} bottles of beer on the wall'.format(amount=50) # the new way
• '{amount} bottles of beer on the wall'.format(**{'amount': 50}) # unpack dictionary (handy trick)
• string.Template('$amount bottles of beer on the wall').format(amount=50) # the verbose way
• string.Template('$amount bottles of beer on the wall').format({'amount': 50}) # no need to unpack. See Template Strings for more information

As you might guess, there are a lot of templating engines available for Python. In my case using one seemed a bit overkill and I ended up coming up with a simple pattern that goes beyond what Python provides by default.

Simple Templating Engine

In my case I wanted to use a recursive dictionary structure that contains the context. This is something I construct based on a collection of JSON files. This task is very simple thanks to Python's json module. Unfortunately the tools above break down when trying to access recursive data.

After thinking about it for a while, the solution wasn't that difficult. It did mean I had to use a regular expression but nothing too shady. You can find my rendering function below:

import re


def render(tpl, context):
    """Substitute text in <> with corresponding variable value."""
    regex = re.compile('\<([a-zA-Z.]+)\>', re.MULTILINE)

    def repl(m):
        group = m.group(1)
        parts = group.split('.')
        value = context

        for part in parts:
            try:
                value = value[part]
                value = str(value) if isinstance(value, int) else value
            except KeyError:
                value = ''

        return value

    return regex.sub(repl, tpl)


print(render('<amount> bottles of <bottle.label>', {
    'amount': 10,
    'bottle': {
        'label': 'Amarillo'
    }
}))

The solution above takes a template and fields (a dictionary) as its parameters and returns a rendered template. There is some error handling in place and it deals with a recursive context.

Conclusion

I think the example highlights the usefulness of regular expressions. The expression doesn't look too bad. If you investigate it further, you noticed I've used a group to extract the data out of a match. It's a handy trick. You can even extract multiple groups in the same time.

There are likely other ways to achieve the same result but this one seemed to fit my purposes just fine. This is a tiny bit of an invoice generator we've been working on at our co-op. We're trying to replace our Google Drive based solution with something nicer and programmatic. So far it looks pretty good and might make a cool web service later on even!