Tuesday, January 6, 2015

Thoughts on the Future of Web Development

Since my previous post about React.js during the Summer, I've had time to level up my development skills and reflect. You could say frontend development tools move forward quite fast. The same goes for backend of course.

On Build Tools

During the past few years I've moved from Grunt to Gulp and Browserify and then onto Webpack. The smart guys have stuck with Make. You can even write simple configuration directly to package.json scripts field but I tend to do that only in panic. My sweet spot is a combination of Gulp and Webpack.

Grunt - It's Magic

The problem with Grunt is that it's filled with magic. That's never good. You don't want to have to maintain a three hundred line Gruntfile. It's doable but not particularly fun.

Gulp is a step towards something better. After all it's just about piping. You can pick it up in ten minutes. Gulp isn't without its issues but it's a significant step ahead. Given it's just JavaScript, you can always hack it if the going gets too tough.

Browserify - a Step Ahead

Compared to RequireJS, which I blogged about years ago, Browserify is a step ahead. The primary benefit over RequireJS is the fact that you can continue writing code in CommonJS module format. You can also directly hook into NPM infrastructure which is a massive bonus.

Webpack - The Holy Grail?

Webpack can be considered the next logical step. What if instead of bundling just JavaScript you had a tool that could bundle pretty much anything including CSS, LESS, SASS, CoffeeScript, Jade, whatnot? Well, this is exactly what Webpack is meant for. Instead of having to build configuration in your Gruntfile or Gulpfile you can just let Webpack deal with it.

You still get the goodies Browserify gives you (NPM, bundling) and then some!

It can even create bundles per page for you. No longer you are forced to download everything on the first load. Instead it will split up the source appropriately and provide partial loads. This can improve site performance massively especially if you have a lot of dependencies split on multiple views. Pete Hunt's guide to Webpack covers the basic approach. I'm aware you might be able to achieve something similar using other tooling but so far this seems very novel approach to me.

Webpack works very well with Dan Abramov's react-hot-loader and takes your React development to the next level. Developing using good tooling makes you work so much faster it's almost unbelievable.

As I'm not an absolute guru with Webpack yet I prefer to use some other tooling, such as Gulp, to copy distribution files around. No doubt there are ways to manage just with Webpack, though, but right tools for right tasks and all that.

The Future

It is difficult to say where build tools might be moving. Perhaps yet another tool comes out and kicks Gulp in shins in turn? Too early to say. Webpack seems to solve the biggest issue for me and no doubt will become more popular as people discover it.

Hot loading might become more than just a development goodie. At some point people will start doing hot loading in production and people will receive updates to JavaScript as they use the app without having to reload. No doubt that opens new cans of worms but in theory that sounds very fun!

On Libraries and Frameworks

On library/framework side the route has been from jQuery to Angular and finally React and friends. You could say jQuery is the PHP of JavaScript. It gets the job done and it's everywhere. Unfortunately it doesn't scale that well for larger scale development. If you need to spruce things up a bit on a static site it's a good pick but I wouldn't develop a JavaScript driven site using it if I can avoid that.

Angular over jQuery

Angular can be considered the next logical step over jQuery and you can even use them together. Sometimes you might want to avoid jQuery altogether. In fact often it's quite trivial to implement something in Angular that would require a yet another plugin in jQuery. I would say Angular is a very good fit for small projects and prototypes. I have my doubts about scaling.

Given Angular is a framework it provides tons of functionality. The problems begin once you hit the boundaries. What if instead of loading each and every dependency the way Angular expects you want to start loading them dynamically per page? Let's just say you have just found a world of pain.

This is a recurring theme during explorations to the Angular world. It works just fine until you hit some sharp edge and hurt yourself. Particularly directives hide a lot of complexity and it's easy to get them wrong. Performance-wise watchers can bite you and you will need to be very careful with them. It may be better to avoid them altogether and consider some alternative approaches.

In fact people are experimenting with ways to simplify Angular development by borrowing ideas from the world of React. Two-way binding isn't your friend always. Some might even go as far and say it's an anti-pattern and I agree. If you can get something done with one-way binding, prefer that to two-way. It's just less headache for everyone.

React.js over Angular

The lack of two-way binding without helpers is one of the strong points of React. Given the flow goes to one direction, it is easy to reason about. As vanilla React deals with just the view portion of an application, you will eventually run into questions like how to deal with models, how should I share data between my components and so on.

To make React shine, you will need to complement with a couple of libraries. I've stuck with react-router, axios (http client) and Reflux. In a future stack I might replace axios with a Swagger client that will generate the frontend API for me automatically. I'll get back to Swagger in a bit.

Flux Architecture and Reflux

The Flux architecture starts where vanilla React stops. In short its implementations and derivatives allow you to scale up from mere components. I have found Reflux a particularly light and smart implementation. It answers to the problems I highlighted very effectively. The idea is quite simple. Your components may trigger Actions. Actions in turn modify Stores in some way. The state of stores gets propagated to your components and the cycle is complete.

Let's say we're modeling selection. I would define a SelectionAction and a SelectionStore. SelectionAction would contain actions select/deselect (accepts item to select/deselect). SelectionStore would maintain the state and on change let components know it changed. In vanilla React the state would be within components themselves. Here we have effectively extracted it out.

By default we are dealing with Singletons so to avoid sharing state you would have to create separate instances but this is more of a special case. Another thing to keep in mind is that if you want to deal with asynchronous operations (say backend query), you should implement these in a particular way.

Reflux provides a preEmit hook for this purpose. In case we implement a basic operation like fetch to initialize our Store, we would define three actions: fetch, fetchComplete, fetchError. fetchComplete and fetchError would then get triggered within the preEmit hook of fetch depending on the result. This in turn would cause our Store to either populate itself or deal with the error somehow.

Error in turn could be passed to some other place, say ErrorStore. You could then listen to that and show the errors to the user, log them and so on. The approach has definite power in it.

Of course you would have to play around with React and Reflux to appreciate the approach. Initially it might feel that you are writing a lot of code for nothing but that's not the point. The goal here is not to minimize the amount of code written but rather to make it easy to follow and reason about. This is something that can get obscured in the Angular and jQuery world unless you are careful.

The Future

It feels like React and Reflux are steps towards a better future. So far I haven't had to worry about performance when dealing with React. There have been gotchas of course and the way you need to think in is quite different than what you might have gotten used to. The approach forces you to keep your entities small and pushes towards components. The cognitive load for creating new components is lower than in Angular as there are less concepts to worry about.

One of the main benefits of React is that it allows you to develop isomorphic JavaScript. Initial attempts have been made to allow Reflux support isomorphism as well. Instead of throwing a bit of HTML and JS to the client and expecting the client to construct the UI using JavaScript, in isomorphic approach we let the backend render HTML and perform initial queries needed. The frontend will then continue from this.

It's back to the same old but this time we are better prepared and gain benefits from the both world. Performance is better and SEO is improved. In a world where latency and poor SEO means lost sales and poorer visibility, what is there not to like?

If you want to take a peek at post-React world, you should study Cycle. There is still room for improvement and perhaps React was just a start. It would not surprise me a lot if it started feeling obsolete within a year or so.

On Backend

As frontend development has become more prominent during the past few years, the purpose of backend has changed. Now it's more about providing a sane API for frontend. RESTful patterns, HATEOAS and whatnot have appeared. You still have to deal with some basic concerns here such as authentication, authorization, business logic, validation and databases. In addition it would be awesome if there was decent documentation available.

Swagger - Definition for Your API

Lately I have been benchmarking Swagger. It is a definition that builds on top of JSON Schema. In short Swagger can be used to describe your API. Various tools can be developed on top of this description. For instance you get interactive API documentation and frontend API client for free.

Depending on the tooling you choose there is of course actual work to do. You will still need to deal with plenty of concerns but using a definition such as Swagger has potential to simplify work. Using a tool like this avoids the pain of having to maintain documentation that is separate from your API. You could of course generate one based on an existing API but that's still extra work that can be avoided.

Furthermore the approach has potential to simplify validation a lot. Given each data model is described in JSON Schema, you can validate against the same schema in both frontend and backend. If the schema changes, the code doesn't have to change necessarily. In ideal world migrations could be generated based on schema changes (JSON Diff?) and propagated to database automatically. In simple cases even databases could be generated without having to maintain duplicate definitions.

The Future

It would not surprise me a lot if usage of definitions like Swagger became more common. Especially when you are working alone or in a small team, you will want to avoid waste. Tools like this have a great potential to do that and allow you to be more agile and responsive towards changes.

From frontend point of view having an API definition simplifies things as it means you don't need to maintain a separate API client. You just generate one based on the definition. Furthermore the definition gives you something to fuzz with. This in turn can be used to improve API quality and security.

Conclusion

Web development moves forward fast. It doesn't take long for new technologies to appear and old ones to stagnate. In a couple of short years we've gone through a couple of build tools and there is no end in sight. I do wonder what on earth could replace Webpack and how?

Just when it looked like Angular had "won", backwards incompatibility of Angular 2.0 was announced. I have a feeling that might have stolen their thunder especially given the release date is still about a year away. In the meanwhile library based approaches will have time to evolve. I would bet on React and friends.

On the backend side approaches like Swagger seem very promising. They take away some complexity while providing a lot if you have patience and time to write out a formal definition for your API. You will have to do that eventually so why not to start with it? This doesn't answer to the problem of API evolution but it's a starting point and much better than nothing!

Sunday, December 28, 2014

HTML5 Canvas Gradients - Animating Gradients

In my previous post I showed you how to generate something I called lattice gradient. That got me thinking. Wouldn't it be cool to animate this sort of thing? Apparently yes.

Demo

See demo below. Put some Quincy Jones as background music and tweak the values of the script for extra hilarity.

JS Bin

Code Example

Here's the full code (not visible in RSS):

I am quite certain the code can be optimized a lot. Now it does plenty of ineffective things. Some lookups could be memoized or even precalculated and so on.

Wednesday, December 24, 2014

HTML5 Canvas Gradients - Lattice Gradient

Inspired by my previous post about creating a rectangle gradient using HTML5 Canvas I started thinking would it be possible to generalize the approach without too much effort. As a result I tweaked the algorithm little and made it accept a matrix as an input.

I guess a visualization like this could be potentially useful for something like heatmaps if you bump up the resolution a lot. No idea of the performance, though.

Demo

Enjoy(?) the demo below. I think you could animate it with a little effort and makes your own x-mas lights or something.

JS Bin


Code Example

Here's the full code (not visible in RSS):

Tuesday, December 23, 2014

HTML5 Canvas Gradients - Rectangle Gradient with Arbitrary Corners

I recently found a question on how to generate a rectangle gradient with arbitrary corners on Stack Overflow. The problem is simple but there is no direct way to achieve this through the Canvas API.

Fortunately writing a generic solution is not that hard. I already did something like this in case of a triangle gradient. This is just a more specific problem. I've outlined the solution below.

Demo

Here is a simple demo for you to play with:

JS Bin

Code Example

And here is a full code example:

We just render a linear gradient per each line while interpolating the colors. Simple as that.

It would be possible to generalize this further to support more color stops etc. but I will leave that as an exercise for the reader.

Sunday, November 16, 2014

Linkdump 23 - Business, Personal Development, Software Development...

Time to dump. Previous one was at June so there is a lot to cover!

Business


Startups


Personal Development


Computer Graphics


Software


Software Development


Big Data


Languages


Productivity


Agile


Editors


Web Development


JavaScript


CSS


React


Angular


Development Tools


Design


Games


Hardware


Art

Sunday, October 26, 2014

Afterthoughts - Tampere Goes Agile '14

What would a year be without Tampere Goes Agile? To paraphrase this years motto "we're all in this together". This was visible in talks throughout the day.

Compared to the last year the event was a bit smaller (~135 vs. ~ 235 people). Despite this the event had some very high quality content some of which I'll highlight next. Perhaps the most interesting thing about this year was the inclusion of multiple workshops in middle of the day.

Considering Tampere Goes Agile is a free event for participants the quality of it keeps surprising me year after year. Besides excellent selection of talks you get to meet a bunch of great people.

Woody Zuill - Mob Programming

The day started with a keynote by Woody Zuill. Initially I was a little skeptical. How could you possibly develop software in a mob? Apparently quite well!

Traditional Split

Traditionally software development has relied on division of labor and expert work. The problem is that once you start to split work you will end up with all sorts of dependencies. This in turn can lead to development blocks and wasted time as you are forced to wait or just do something else while at it. 

Multitasking just makes the situation worse. You will end up jumping between tasks while waiting for answers. This in turn creates stress that definitely doesn't help.

Get Together?

What if all the people that have the answers would be in the same room at the same time and developed the software together? You could invite domain specialists or the client even to participate in your sessions. Even though this might sound a little strange approach it alleviates those concerns I just highlighted.

When you are working as a mob this allows you to focus all the energy on a single task at a time (automatic single piece flow!). If you end up blocked there is someone with the answer in the vicinity. It is like the concept of pair programming scaled to the team level.

Woody's Approach

In Woody's configuration there is a single driver (rotated every 15 mins or so). That person writes the code. The rest operate as navigators and guide the driver. The primary advantage of this seems to be that while the driver can think in code the rest can focus on thinking in human terms. They might also spot inefficiencies and help guide the driver towards better technical solutions that might not have been found otherwise.

Given there is so much expertise in the room at the same time this might allow the team to avoid entire categories of problems or eliminate some of the existing ones! Multiple perspectives seem to allow this and the approach helps to avoid some of the traditional sources of waste in software.

The approach isn't without its challenges. It could prove challenging for certain people as it literally brings your work out in the open. Working directly with other people isn't the way software is normally developed. For work like this to succeed you will need to remember to be kind, considerate and respect others even if they might be wrong or irk you in some way.

Besides removing waste the approach allows information to be shared more effectively (reduced bus factor). As it encourages information sharing through collaboration people pick up new skills faster. This is something you simply don't get in a segregated environment prevalent in the industry.

Takeaways

Mob programming is something Woody and his peers discovered through introspection. It was not just something that came out of thin air. Rather they ended up developing software this way because it felt like the right and most effective thing to do. To quote Woody you should find your own way. That said, maybe mobbing works out for you but don't expect it to be a silver bullet.

Per Jansson has written an introduction to mob programming that might give a better idea of what it's about.

Making Work Flow - Lauri Hahne

The gist of Lauri's talk was in how do traditional approaches fail us. He sees software development as a systems and a learning problem. This is very opposite to the waterfall view. I agree with Lauri here. The system, context in which we develop software, puts heavy constraints on productivity. Unless you are developing something you have done before exactly, learning will need to happen. Both of these contribute heavily to project success even if you aren't aware of the issue.

The Traditional Model

The traditional development model is very static in sense that it builds on things like requirements management, HR and distribution of work for maximum utilization. The traditional model focuses heavily on aspects such as technology choices, finding proficient people and division of labor. Teams are formed per project, work is performed on individual level and all of that is managed by a project manager.

If you are building something you have developed before I have no doubt the traditional model couldn't work. What happens, though, when you need to work on something more complex? How do you know you are building the right thing in the first place? How do you measure the impact of your results?

The traditional development model has a lot to owe to Taylor and the industrial era. The theories expect that people are lazy by default and need to be managed for work to get done. Specialization is required to get most out of division of labor. What if some, or all, of the assumptions we've built our development processes are wrong?

What Are Good Teams Like?

What if people aren't lazy and actually want to get things done? What if good teams are larger than the sum of their parts? What if it is beneficial for people to have cross-competencies, not just specialities?

According to Hahne teams that finish early, accelerate faster. He highlighted the importance of stable teams, swarming (single piece flow again!) and scrumming the scrum. You have to be prepared to work on the way you work for things to improve.

You will likely have to discard some of the older ideas to get better. There are plenty of sources for new ones. Of these Hahne mentioned Toyota Production System (Kanban etc. come from here) and John Boyd's Blitzkrieg. He also suggested rethinking division of work. What if you operated per technological layer, screen or feature as a team. It's not a huge leap to mob programming from here.

Hahne values stable teams of teams formed per project. Stable teams operate more consistently, eliminate HR and allows them to operate more effectively as you minimize the effect of Tuckman cycle related to forming of teams.

Hahne highlighted swarming as it will allow each team to operate on a single task at a time (single piece flow). Rather than working on tons of smaller issues at once you actually focus on getting something ready. This eliminates a significant amount of waste and multitasking which in turn improves productivity.

Focus on Process Improvement

Hahne's focus on working on the process itself was prevalent. You should have a good idea of what is blocking team's performance and take steps towards eliminating it. Again, focus on teams seems to be the key here.

If software development has taught me anything, it's that errors pile up and accomplishments disappear. When things go south, people start feeling bad and the cycle gets even worse. If you rush things to make a deadline, you will pay for it later in forms of technical debt and reduced morale. What if you could do things better in the first place and avoid this?

Takeaways

Traditional software development puts too much emphasis on artefacts that have been derived from the industrial era. What might have worked for some tycoon a hundred years ago doesn't quite do the trick in modern, dynamic world.

I agree with Hahne in that teams are a suitable smallest unit. Split work per teams and don't split up teams. Pick new concepts as needed and remove those things that block team success. It is not like you can build a superb team over night. It might form over a longer time, though. Invest in your diamonds.

Trying to Change Company Culture is a Fool's Errand - Sami Honkonen

I have discussed company culture earlier. Sami's talk gave good food for thought. How are cultures formed? How can you change cultures? According to Sami in order to change a culture, you will have to adjust system conditions that cause it. In short culture should be seen as a result of all the things that are going on, not something you can tweak directly.

Context is the King

If you want to change a culture, you will have to adjust the context. If your cake (culture) tastes bad, you will have to bake it differently using possibly different ingredients. Maybe your cake (culture) will be better that way.

How do you know what to tweak then? It turns out system conditions are everywhere. Sami gave a fish tank as an example. Just to give you some idea, at least the following conditions affect the system: lighting, oxygen level, amount of fishes, race of fishes, amount of space in the tank, cleanliness, nourishment, decorations and so on. You can think up a lot more.

How to Change Company Culture?

It is possible to come up with a similar list affecting a company culture. In order to change the culture you will have to change these factors. As an example let's say your company is one of those cubicle farms. How do you think it will affect the culture? The work is likely done in isolation and the people rely on email a lot. The culture is focused on individual performance. This in turn prevents working with others. If you wanted to change this culture, you would have to change cubicles into something else.

Another example is limited WIP (work in progress). If you have the system set up based on teams as Hahne suggested and enforce a WIP limit of one, it gives the teams a very high incentive to focus on finishing. If you take this further and provide incentives based on team performance, you will find that it will be difficult for teams to collaborate with each other.

Takeaways

Sami's talk described well what culture is about. It's something that exists as a result based on system conditions. If you change these conditions, the culture will change as well. There is no direct way to modify it. Rather you tweak it indirectly like this. Even small things might have surprising effects and I expect it's a whole can of worms if you really start thinking about it.

In short if culture eats strategy for breakfast, system eats culture for lunch.

Introduction to Retrospective Facilitation - Ville Ruuskanen

Isn't nostalgy great? Who doesn't love 80s clothing and music? Thankfully retrospectives don't have to go that far and you don't have to wear those tight 80s clothes to facilitate them.

Why to go Retro?

The primary advantage of running retrospectives for your team is that they provide a great chance to perform introspection and think about what works and what doesn't. This ties back to Woody's and Lauri's talks in which they highlighted the importance of process improvement. You shouldn't run a retro only at the end of the project. Rather the greatest benefits are gained if you measure the pulse of your team continuously. In Ville's case he seemed to prefer a longer one (2-3 hours) once every two weeks though your mileage may vary.

There is no single right method to run a retrospective. Instead you should thrive to vary their content. Ville provided a rough frame in which you can build the event itself. I won't go through the details as you can find them in the slides below.



Given there was another workshop running in the same space at the same time, it was a little noisy at times. Apart from that it was just great as I didn't know much of the topic beforehand. If I ever have to manage or lead a team, I will have yet another nice tool at my arsenal.

Self-Management, practical tips for your personal journey - Erik Anderson

In order to remain competitive in the current business environment, you will have to continuously push yourself forward. This was the topic of Erik Anderson's talk. Curiously self management is one of those topics they don't teach you at school. Rather you are expected to pick it up yourself and some people just fail to do that. If you are aware of the issue, you can probably do something about it.

Erik split the topic in three overlapping parts: body, brain and society. We can do something for each up to a limit. Especially the society part tied back to earlier talks. Where and how you work affects your well-being directly whether you acknowledge that or not.

Body

It is important to keep in mind that humans used to roam savannah. Modern life as is is quite recent invention and we certainly haven't adapted to it perfectly. This in turn causes all sorts of problems for bodies designed for something else.

If we keep these constraints in mind, we can understand better why exercise is so important yet undervalued. Being in a good physical condition affects everything. In addition effects of sleep and proper diet should be kept in mind.

Brain

Interestingly exercise gives your brain what it needs in order to change (improved learning, memory). It is possible to "hack" brains in various ways. Sadly they don't provide user's manual with them. I guess we wouldn't be so lost otherwise.

Mindfulness, being aware of what you feel, allows you to observe what you feel rather than react. If you train the brain through meditation, it will get stronger just the way body does when you learn it.

It pays off to understand how habits work and what happiness means from perspective of brains. Learned helplessness/optimism is another of those concepts people should be aware of.

It is easy to end up in a cycle of depression because depression causes more depression. Breaking the cycle is the problem. On the other hand it might be possible to gain optimist traits and get into another kind of cycle. In the end it comes down how you deal with adversities in life and react to those.

Society

Erik discussed topics of personality styles, personal strengths and team profiling. I know personality tests and such sound like pigeonholing but the fact is that each of us has their preferred way of working. It is important to be aware of that as that is something that contributes to our happiness.

If you can adjust the way you work to suit your personality better, you will likely achieve better results and be happier while at it.

The way we relate with society has a big effect on us. Erik described the concept of transactional analysis. The key thing to understand here is that a person is a multi-faceted concept in how it relates with the world. These facets (parent, adult, child) get triggered in social situations and helps to explain why responses are the way they are. If someone snaps at you, maybe there was something about the way you poked the person that triggered the child response to parent question?

Takeaways

Erik's talk showed that there is a lot to learn about. Self management and improvement are large topics. To quote Erik you will have to keep on tweaking things to see if they get better and be open for changes. That is the way forward.

Conclusion

It's probably not an understatement to say that this Tampere Goes Agile was the best one so far. I loved the talks and even the party after the event was fine although I felt embarrathed over wordings of certain agile songs performed. I hope the event will be arranged again next year and definitely plan to go it if is!