Read my book

I wrote books about Webpack and React. Check them out!

Wednesday, February 25, 2015

How to Publish and Maintain NPM Packages?

It is almost amazing how popular NPM is these days. At the time of writing it has whopping 127k packages! It is useful beyond Node.js and people use it increasingly on frontend side as well. Tools such as browserify and webpack can hook into packages hosted on NPM. You can even build a package manager on top of NPM but that goes sort of meta.

As I have been publishing and maintaining NPM packages for a few years I thought it might be a good idea to document some of my practices. It is a simple system to use as long as you are aware of a couple of tricks.

Initializing a NPM Project

Technical Difficulties by Wonderlane
(CC BY)
NPM relies on a configuration file, package.json. It is placed on the project root. NPM cli provides a handy utility for generating it, npm init. Simply answer the questions and you should end up with a package.json. Don't worry about all the questions, you can tweak the file later.

Introduction to Mankees

When I'm starting a new project I like to cheat a bit. Years ago I developed mankees, a little tool that makes it easier to script on top of Node. It comes with a little package manager that has been developed on top of NPM. It just does its best to hide this fact.

One of the first script I wrote for the environment was known as init. It is a simple tool that can generate project scaffolding for you. It parses ~/.mankees/config.json and then injects those values to Handlebars templates of the project you want to create. It is simple enough to create your own templates.

I have set up basic README.md, package.json, LICENSE, .travis.yml and .gitignore in my basic Node template. It can be tedious to set each by hand so this saves some effort. All I need to do is to hit mankees init node {project_name}. It will create a directory for me with basic details set up. After that I just need to code, set up GitHub and publish to NPM.

Other Scaffolding Tools

I know there are more powerful scaffolding tools such as Yeoman. For a simple Node package they seem a bit too much. I rather take something simple and add than take something complex and remove. Less effort.

Set Up Version Control

After you have set up your basic project you should hook up Git. The basic steps include git init, committing your work as an initial commit and pushing the work to some repository (ie. set up something at GitHub or Bitbucket).

Publishing to NPM

Node packages come in all shapes and sizes by Diana Schnuth
(CC BY-NC-SA)
package.json contains actually quite much information. Nodejitsu's interactive guide gives you a good idea of what each field does. You should aim to fill the most. It is particularly important you make the main field point at the entry point of your package.

If there's a cli script included, you should set bin. If the cli command name matches your package name, you can enter a string there directly, otherwise you should use an object.

As NPM won't allow multiple packages with same name, you should check out https://www.npmjs.com/package/{your package name} before settling on one. Sometimes this can be the most difficult part of the project as many common names have been already taken.

At times people like to name their Git repository with node- prefix. This is less ambiguous than just sticking to a package name. The package will still retain its short, Node specific name.

Testing Configuration

You can test your main and bin configuration by using npm link. This will make your package available through Node environment and you should be able to access it anywhere. Just require('yourproject') within Node console or try to hit cli command(s) if you set it up.

Before publishing anything it can be a good idea to tag a release and update package.json. As doing these steps manually is utterly boring, there's a little utility for this. Simply hit npm version {version} (example: npm version 0.1.0). This will perform the steps for you and create a git commit with the version.
If you haven't registered to NPM yet, you should set up an account at NPM site. To make the cli aware of this, you should use npm adduser.  In case you want to share authorship of a package with someone other, you should use npm owner.

Publishing a Package

The next step is the one you have been waiting for. Hit npm publish and your package should appear to the registry. You can verify this by checking out https://www.npmjs.com/package/{your package name}. Besides this you should remember to hit git push and git push --tags.

Maintaining a NPM Package

There are a couple of simple things to keep in mind when maintaining a NPM package. It will make it a lot easier for you if you respect the semver. This will make it simpler and safer to consume the package.

Publishing Something to Test

In case you want to publish something for public to test, you can do this in two simple steps. First hit npm version {version}-beta. After that publish like this: npm publish --tag beta . npm install will still point at the stable release. To install beta you would hit npm install {your package name}@beta. You can of course vary the naming and be more specific but this should give you the basic idea.

Types of Dependencies

Dependencies by Linux Screenshots
(CC BY)
NPM packages come with three kind of dependencies. Direct dependencies, devDependencies and peerDependencies. Direct ones get installed with your package.

Development dependencies are something that you are meant to use only when developing the package itself (ie. testing utilities and such). Node will install both by default. You can avoid fetching development dependencies by hitting npm install --production.

Peer dependencies are most lenient of these. Suppose you have a plugin but you would rather not depend on the environment directly. In the worst case you could end up with a project that has multiple different versions of the host environment due to dependency declarations. That's definitely not good.

peerDependencies solve this problem. By setting up a peer dependency you defer the problem to a higher level. If you are developing for instance a React component, this would be the right way to go.

Dependencies come with some further complexity, namely dependency version declarations. NPM defaults to caret (^). In addition it is possible to use tilde (~). Given this topic can get rather complicated fast, you should study node-semver with care. That's where it all stems from.

Sometimes it may be handy to point to some dependency directly (say it's under development, not at NPM etc.). In case of GitHub, you can simply state a dependency version like this: {github user}/{project}#{reference}. Reference is optional and may be commit hash, tag or branch.

If you want to exclude certain files out of your distribution version, set up .npmignore. You may also find it useful to utilize NPM hooks. Those allow you to do things at various steps (ie. before publishing, after installing and so on).

Dealing with Scripts

Note that if install some testing tool through devDependencies, you can point at it directly within scripts section. Ie. in case of webpack you would do something like this:
  1. npm i webpack --save-dev (--save works too. You'll have to add peer dependencies by hand!)
  2. "scripts": {"build": "webpack"} at package.json
  3. npm run build - This works because NPM will add webpack build tool to the PATH temporarily when you hit npm run.

Updating Project Dependencies

As part of maintaining is about worrying about dependencies, I have set up a mankees script for that purpose. mankees update_deps bumps up project dependencies for me. It is bit of a nuclear option but it has been a great timesaver for me. npm-check-updates seems like another good alternative.


Services Helping with Maintenance

There is a lot to worry about when developing packages. It gets only worse when you get to frontend side since then you may have to support multiple different environments. I won't dig into that, however, as Alexey Migutsky has done so in detail. Instead I'll link you to a various of services to check out and apply as you feel necessary:
  • SauceLabs - SauceLabs runs your frontend tests (Selenium etc.) against multiple browsers. Free for open source. In addition they have a badge available you can include at project README.
  • Travis - Travis is able to run your tests against multiple Node environments. This can reveal issues with specific versions. Again, there is a badge available and Travis will be able to check GitHub PRs automatically.
  • David, VersionEye, Gemnasium - These services are able to check your package dependencies and give warnings accordingly. There are badges available. I have been using Gemnasium myself. It gives me a weekly digest. In addition it warns about possible security issues.

Conclusion

I hope this post gave you some idea how to deal with NPM packages. In the end it's not complicated once you learn the basic commands. The hard part is in figuring all of this out.

It would be interesting to hear what sort of workflow and tooling you use to make it easier to develop and maintain Node packages.