Book Review - Instant IntroJs by Ehsan Arasteh and Afshin Mehrabani

Disclaimer - I received a review sample of the ebook from Packt Publishing

Packt has a specific line of books known as Instant. I reviewed one of those, namely Instant Zepto, a while ago and wasn't totally impressed. The value just wasn't there. This time I'll try again with another book of the same line.

Introduction

Instant IntroJS is a book by Ehsan Arasteh and Afshin Mehrabani of Usablica, incidentally authors of Intro.js.

I was aware of the library before reading this book. It happens to be the leading entry at its JSter category of tours and guides so they must be doing something right. Technically the library isn't that complex but it solves a tangible problem.

These kind of libraries make it easier to explain what your site does and help users adopt new functionality. Using one can help you to optimize conversion. They aren't used that commonly yet but every once in a while you see some refreshing use.

Does a library this small need a book? A decent programmer can work out the details by examining the available examples and documentation. In Intro.js case you have merely some examples available but given the library is not that big itself, it is fast to skim through. You could say this book fills the need for documentation if there's one.

Content

The book is quite short as you might expect given it's "instant". Out of 48 pages roughly 30 have been dedicated for content. Within those pages you'll find a beginner friendly overview and introduction and get to understand the basic ideas behind the library. In addition integration with frameworks such as Rails and Yii are discussed. There is also some information on localization and RTL (right-to-left).

I probably would have left some absolute basics out. For instance building a project should be trivial. And even if it is hard, it should belong to developer documentation. I was also a bit stymied by the fact that event handling is discussed twice. I would have cut some of that out and spent the pages on something more important.

As usual the book came with a collection of code examples available as a separate download. Sadly they were pretty useless in this case. Only one example was complete enough to run "as is" and even then you needed to download and set up some dependencies on your own. This was bit of an issue even with "Instant Zepto" but in this case the issue is somewhat pronounced. I am certain the authors are capable of more.

Formatting of the book was alright expect for a bit low quality images and missing syntax highlighting. It is clear the authors know what they are talking about. The text was clearly written and easy to consume.

Conclusion

Of course a couple of questions remain. Should a book like this have been written in the first place? It feels like something that you could expect to find as a part of library documentation. In this case there would be a great incentive to keep the documentation up to date. APIs change and libraries evolve. Will the book stay current?

Given the relative simplicity of the library it isn't that hard to learn to use. The value isn't there at least for me. I might buy something like this on impulse (say $5 ebook). The current price range ($10) feels a bit much.

In case you are absolute JavaScript beginner and you don't have much time to study and just want to get something done, perhaps the value proposition is there. For an expert programmer it is a bit so-so. You can get a real book with lasting value for $10.

Node.js - parse-env to Simplify Configuration

Some of you might remember my blog post about Node.js configuration patterns. As I have had to do quite a bit of Node.js development lately I have learned to appreciate the utility parse-env, a tool of mine provides. It merges file based configuration with environment based one and provides the results through an easy to use module.

I just made a couple of small changes that improve the utility of the tool further. Now it treats uppercase words properly (ie. instead of parseJSON -> PARSE_J_S_O_N you get PARSE_JSON). In addition it is possible to disable warnings about missing environment variables by setting MUTE_PARSE_ENV true.

Yeah, it's a tiny tool. But it can potentially simplify your Node.js development at least a little bit so give it a go!

webshotter - Capture multiple screenshots per website

webshotter is a small Node.js utility relying on PhantomJS and ImageMagick that makes it easy to generate collections of screenshots of websites. There isn't much else to say really. I guess writing this utility showed me how easy it is to throw something together using a couple of Node.js libraries. The utility itself is still in early stages of development but it gets the basic task done quite nicely.

Continuous Delivery - Getting Started with Node.js and Wercker

I have to admit I'm not that strong at DevOps related things. I would probably make a terrible server admin. Fortunately things have become easier to understand during the past few years so even if you weren't born with a silvery Unix spoon in mouth, you can still get something simple working. In this post I will try to go some basic concepts and share my experiences with one particular service I have been trying out.

When it comes to deploying there are multiple approaches. The old skool way is simply to FTP the files on the server on change. Once you get bored of that you might maybe set up rsync (keeps files in sync automatically). Or you could use some version control based scheme in which the server pulls the changes on deploy.

These simple ways might be more or less alright in limited usage. At some point you might run into a wall, though. And managing the server may become cumbersome. What if you need to revert some change for instance? There are better ways.

Continuous Integration and co.

Continuous Integration and associated concepts have made a big impact on the industry during the past few years. The value of these concepts lies within automation and the visibility they give to the development process.

Your Continuous Integration server may execute your tests, some of them heavy, after each push. This means you will be able to catch possible regressions earlier and fix them cheaper.

Continuous Delivery builds upon this idea. The idea is that should the tests pass, you may deploy the build manually. The process will then copy your build to the server and rig it up.

Continuous Deployment actually goes one step further and automates this step. That takes some serious trust in your code!

Setting Up Continuous Delivery Using Node.js and Wercker

The Wercker guys do a great job at explaining how to set up a continuous delivery setup using their system. It is currently in free beta and well worth checking out. They support a variety of platforms and make it very easy to integrate with services like GitHub. I used Digital Ocean for hosting when testing.

Digital Ocean is one of those affordable options when it comes to hosting. Remy Van Elst goes through the pros and cons of DO at his blog. Note that even though Digital Ocean shows prices in both monthly and hourly charges, it is important to note that the hourly charge might not be the same you could expect based on services like Amazon's EC2.

Even though your server is in "off" state, it still gets billed! This has to do with the fact that DO still keeps the IP and hardware reserved for you. As long as you are aware of this fact you should be fine, though. The price is hard to beat.

As you can see based on the Wercker tutorial, they use a YAML based configuration scheme for defining how to build your project and how to deploy it. There are various base boxes to choose from. Each box comes with various dependencies set up already. For instance Node.js developers may want to use `wercker/nodejs` box.

In order to give you some idea of how to set up a basic build with Bower, Grunt, SASS, NPM and all that jazz, consider the Wercker configuration below:

box: wercker/nodejs
build:
    steps:
        - script:
            name: install compass
            code: sudo gem install compass --no-ri --no-rdoc
        - script:
            name: install grunt
            code: sudo npm install -g grunt-cli
        - script:
            name: install bower
            code: sudo npm install -g bower
        - script:
            cwd: frontend/
            name: install npm dependencies
            code: |
                mkdir -p $WERCKER_CACHE_DIR/wercker/npm
                npm config set cache $WERCKER_CACHE_DIR/wercker/npm
                sudo npm install --save-dev
        - script:
            cwd: frontend/
            name: install bower dependencies
            code: bower install --config.storage.cache=$WERCKER_CACHE_DIR/wercker/bower
        - script:
            cwd: frontend/
            name: build project using grunt
            code: grunt build
        - script:
            name: echo nodejs information
            code: |
                echo "node version $(node -v) running"
                echo "npm version $(npm -v) running"
deploy:
    steps:
        - add-to-known_hosts:
            hostname: $SERVER_IP
            fingerprint: ff:ff:ff:ff:ff:ff:ff:ff:ff
        - mktemp:
            envvar: PRIVATEKEY_PATH
        - create-file:
            name: write key
            filename: $PRIVATEKEY_PATH
            content: $WERCKER_PRIVATE
            overwrite: true
        - script:
            cwd: frontend/
            name: transfer application
            code: |
              pwd
              ls -la
              tar czf - * | ssh -i $PRIVATEKEY_PATH -l root $SERVER_IP "cd /var/local/www; tar xvzf -"
        - script:
            name: start application
            code: |
              ssh -i $PRIVATEKEY_PATH -l root $SERVER_IP "if [[ \"\$(status node-app)\" = *start/running* ]]; then stop node-app -n ; fi"
              ssh -i $PRIVATEKEY_PATH -l root $SERVER_IP start node-app

Yes, I admit it's a bit chunky. Let's go through some basic concepts to dig into the meat so to speak. First of all I choose to use the Node.js base box provided by the Wercker guys. On my build step I install some dependencies and then build the project. My project isn't exactly conventional as my server source is within `frontend/` directory. This is the reason why I use `cwd` every once in a while.

Some of the data is cached so it is faster to build the project later on. You would normally want to execute your tests in the build step too. In this case I was just interested in getting the basic flow to work.

On my deploy step I first do some configuration necessary to make the communication between Wercker and my server to work. In this case I just point to the server by ip although you might want to use a real domain instead. In case you are wondering about that fingerprint bit, it is possible to generate it like this:

• ssh-keyscan -p 22 -t rsa hostname > key.pub
• ssh-keygen -l -f key.pub

It is a security feature that should help to avoid MITM attacks if I remember correctly. Anyway, that's something that is easy to set up and hence you should do it.

After I have the connection related issues sorted I actually transfer the data to the server using a tar pipe. This is a trick I learned from some version of Stack Overflow. It speeds up transfer immensely especially if you have a lot of small files around. I recommend giving it a go just to see how powerful technique it is.

Once the build has been copied to server I simply execute the server. For this purpose I have set up a simple upstart script. The nice thing about it is that you don't even need a supervisor like forever or monit as it will be able to keep your server up should it crash for a reason or another. It is preferable to run your server with a user that has limited rights. That way you mitigate the amount of possible damage an attacker may be able to do to your server.

For some reason the upstart scripts Node.js people like to use seem usually awfully complex. For this reason I asked a couple of friends to provide examples of good ones. Here's one by nnarhinen. Another one from opinsys. Combine and extend based on your needs.

Conclusion

Continuous delivery systems like Wercker make the developer's life a step simpler. It takes some pain out of deployment and allows us to focus on actually getting things done faster. You can never be quite fast enough. There are still some good things, like containers, in horizon. They promise to simplify deployment even further and make it faster to execute tests. For instance you could easily have multiple databases against which to run your tests in parallel. But that's a topic for another post if I ever get into that world.

I hope this post inspired some of you to give some system a go! Please share your experiences in the comment section below. It would be very nice to hear what sort of setups you use in practice and how you benefit from them.

Afterthoughts - Open Knowledge Roadshow 2013 - Jyväskylä

One of the greatest benefits of internet is that it allows aggregation. For aggregation to happen data needs to be in easy enough format to consume. Government data might be available but in a format that is hard to consume. The open data movement aims to change this situation. This way it can be considered a sibling to open source. This time the focus is on data, though. The benefits of collaboration are the same, though.

We are still in the beginning of the process here in Finland. Some promising moves have been made. A while ago map data came available. It is still missing some routing related data but there's plenty open already. Most data related to public transportation is available as well and the situation will get a lot better within a year if everything goes right. These are just a few examples.

Making government data available makes it considerably easier to build services, and business even, on top of it. Government's own services aren't known for their usability or innovation. Opening data allows the government to leverage the talent of external developers.

Open Knowledge Roadshow 2013

In order to make the situation better, the Finnish branch of Open Knowledge Foundation decided to organize a roadshow. The purpose of it was to make more people, esp. from the government side, aware of the benefits and foster community creation. There are a lot of talented people in Finland. They just need common goals to make anything happen.

I helped to organize the local event and participated in it. It was split over two days around 8th and 9th of November. The first day was more about networking and discussing the concepts whereas the latter focused more on getting things done.

The Potential of Open Data

In order to demonstrate the potential of open data I helped to convert data available in matka.fi Kalkati format to GTFS. This allowed us to showcase a navigator application developed for public transportation. The navigator itself has been developed by Tuukka Hastrup as a part of Code for Europe program. The source is available for those interested.

Unfortunately the recent revision of matka.fi data didn't contain data for Jyväskylä. So we had to use an older data dump instead. This might have something to do with the fact that the current bus service provider will likely be replaced with another one next year. At the same time the responsibility for the data will be passed on to the city so that should help with the problem.

I have been thinking of scraping the data from the company website. It would not be that hard. If there is enough interest I can definitely do that.

Thoughts on the Main Event

There were a couple of talks explaining the significance of open data on local level and potential applications. One interesting application had to do journalism. The director of a local parking garage company had stated that they are running out of space and need to build yet another storey. There was no data to back this up nor would they provide it when asked (considered confidential). They do provide realtime data on their website, though.

You can likely guess what happened next. Heikki Salo wrote a simple script to fetch and store the realtime data to generate time series. Surprisingly the graphs showed that storeys weren't fully utilized. Another storey isn't needed for some time after all. And even the director had to admit this publicly.

Another application was provided by Tuomas Räsänen. In his case he scraped the meeting data provided by the local council (terrible format) and plotted it on a map. Klupu has inspired people at the capital to develop something similar. I guess it's our turn to one up them again.

The last application implemented by Esa-Matti Suuronen on top of weather data provided by Ilmatieteen laitos has life saving implications. Hyppykeli simply displays whether or not it is safe to skydive based on the current wind conditions.

These cases in addition to that public transportation navigator give us some idea of the power of open data. Sometimes it might take some extra effort to make the data usable and open but after you have the data you can build very interesting things.

If the data is available, it is possible to think out of box and implement something like a way for the blind to see. BlindSquare achieves this by combining data from various sources and enabling the blind people to hear what they should be seeing. How is that for technology?

Thoughts on Konstruktori

Whereas the main event was really popular, around 100 people or so, only around twenty or less appeared at Konstruktori, a pre-event of Instanssi. The timing wasn't perfect as it coincided with a holiday so that could explain a part of it. Also the PR work could have been more effectively and we could have made it easier to participate remotely.

We did get some concrete results out of the event. For instance we translated most, if not all, strings of Froide to Finnish. It is a German made application that allows the people to request data from the government. This simplifies the process somewhat and makes it more approachable and you might say democratic. The easier it is for people to demand for information the more likely the government is to make parts of it open given there is a very concrete incentive.

I worked pretty hard on a couple of projects of mine. I restored an old project of mine, jkl-event-scraper. It simply scrapes the event data of Jyväskylä and provides it in a usable format. I modernized the innards and replace zombie with combination of request and cheerio. In addition I implemented a small API and gave access to the events in a calendar format (iCalendar). Particularly latter work is considered experimental.

In another project of mine, it2rest, I wrote an API for a Google spreadsheet I help to maintain. The spreadsheet simply contains some basic information on local IT companies I am aware of. Having an API available makes it much easier to consume. I intend to use the data in another project later on.

Conclusion

I hope the event worked as a catalyst locally. I would love to see more data become available, and more importantly, more applications to be developed. Having data is cool but having actual applications is better.

There are some clear benefits to having government data open and publicly accessible. It encourages transparency and improves efficiency, something governments aren't particularly famous for. I see it as a natural step towards a more participative democracy.