Node.js is not a toy. Checkout the new large scale web application I helped build. signforms.com
We produced a very complex, solid, secure, fast and scalable server solution using node.js in a very short timeframe with amazing results. We used a number of different modules and server side tools to achieve our goal.
What got me excited about this project was the fact that we were tasked with building it in 2 weeks, and had complete technical control over the project.
After looking into what was going to be required for the project, including security, scalability and complex document processing, we decided to use node.js for the server side and a very powerful client side rendering and binding system named gaffa-js on the client side. My colleague has written a blog post about the front end development if you are interested in reading more about that Deploying a web app in 14 days no html
Node.js as a server
I am not a fan of the “web application frameworks” such as express.js being widely used without thinking about what is really required. I find them bloated, and unnecessary in most situations (kinda like jQuery…) and personally I much prefer to use a number of smaller modules to build my own solution.
I have been using Beeline for a lot of my projects lately. As it says on the box, it is laughably simple router which takes seconds to setup. It supports everything from simple catch all routes to very powerful regex matching.
Cookies and Keygrip are 2 modules from Jed Schmidt that again I have been using in many, if not all of my projects of late. Cookies provides a simple wrapper around the getting and setting HTTP(S) cookie headers, and can be used in conjunction with Keygrip to provide additional security by signing the cookie to prevent tampering.
If you are doing anything with passwords or any other sensitive data, and you are not using bcrypt, please, please, please get educated. Here is a good place to start. How to Safely Store A Password
Cryptr is a simple encrypt and decrypt module for node.js that I have written myself for doing simple obfuscation / aes256 encryption on values that need to be decrypted at a later time
I currently use this for a number of items which are encrypted for obscurity rather than any real security.
Formidable is a module for parsing form data, but comes into its own when processing file uploads.
I had not used Formidable before this project but after doing a lot of research and doing a lot of testing, it came out on top as the simplest, fastest and most secure way of handling multiple file uploads.
A lot of time was spent to determine the correct database platform to use for SignForms. In house there was already precedence and knowledge in OrientDB, CouchDB and MongoDB so the decision was around what platform would best suit the type of data we are going to have to store.
While there was a lot of research done and a lot of points considered, a lot of comparisons such as Visual Guide to NoSQL Systems were used as references and eventually we decided on MongoDB. The key decision points for use were:
- Dynamic Queries
- Most likely gain benefit from indexes vs map reduce (although we do have some map reduce requirements)
- Database size will most likely grow large thus distributed partitions maybe be required in the future.
- Performance of large datasets.
- Storing a large amount of volatile data, at least until documents are finalised.
Additionally, Mongoose the node.js wrapper for MongoDB was a breeze to setup and work with making database access simple and fast from our chosen server platform.
Here in lies the challenge. When looking into the requirements for this project the only thing that really scared me was how the hell was going to process and manipulate the pdf files. I spent days working with different tools and approaches to the problem. I hit many a brick wall but also came across a number of insightful discoveries of things I didn’t even know existed.
This was a challenging journey and probably deserves its own post about how these problems were solved. I will aim to have that post up in a few days.