All of this knowledge buildup is for a personal website project that I have planned (and where this blog will call home!). Before I start development on the website directly, I will create a series of prototypes to get a feel for some of the technologies I will use in my websites stack. With these prototypes I can make sure the technology I choose is a good fit for the full project. Also I can use them as templates to complete future discoveries! In general if you have the time to build prototypes with technologies you want to use in a production project it is a great idea!
My planned production web stack will be either MEAN (MongoDB, Express, Angular, Node.js) or MERN (MongoDB, Express, React.js, Node.js). Both of these web stacks share three technologies - MongoDB, Express, and Node.js. I used all three of these technologies to build this prototype!
The prototype is a REST API that lets users look at songs and artists. They can also comment on songs. MongoDB stores this information in a database called
music_api. Databases in MongoDB are simply namespaces, which is much different than their Relational Database equivalents. In a RDBMS you need a username and password among other items to connect to a certain database. MongoDB does not have this requirement.
The data is then stored in collections of documents including songs, artists, and end users.
The Node.js run-time environment and Express web application framework then expose a REST API to users so they can query, create, update, and delete items in the database (all CRUD operations).
For the prototype most of the functionality surrounds viewing and manipulating songs, however in a full application these uses would be expanded. This could include creating and updating users, rating songs, user-to-user interactivity, etc. Now let's look at some insteresting aspects of the prototype and certain challenges that I faced while creating it.
For interacting with the MongoDB database from Node.js I chose to use a module called Mongoose. Mongoose allows you to model database JSON objects and perform all MongoDB queries, inserts, and updates. With MongoDB you don't really need to use a ORM (Object Relational Mapping) framework since all the data is already in JSON form which any programming language can deal with. However, Mongoose gives us some additional capabilities which make it desirable.
First off Mongoose allows you to set strict schema rules for your MongoDB objects. As previously mentioned MongoDB is schema-less, so this stricter model can help restrict what exactly a user can place in a document. You can also create nested schemas for complex JSON documents. For example, my song collection schema contains a nested schema for a list of comments.
Schema() constructor function takes a JSON object that represents all the properties of the MongoDB document. You can also add additional validations, such as making certain fields required (
required: true) or having a default value (
default: Date.now())1. When a user uploads a JSON object to a schema and tries updating the database, only the properties seen in the schema will be persisted to MongoDB.
Also there is a comments property which takes an array of type
CommentSchema can be defined similarly to how I implemented
Another powerful feature of Mongoose is we can define database indexes directly on the
Schema object. With advanced features such as this, Mongoose has transformed into something much more powerful than a ORM. With Mongoose I never used the MongoDB CLI since everything I needed could be performed through Mongoose. Here is an index I made on the
name property in the artist schema.
Mongoose allows you to perform any query, update, insert, or delete operation on a Schema. By default Mongoose uses callbacks to perform these operations. We want to avoid callbacks as they become ugly and hard to read once nested database calls are made (commonly known as callback hell). Luckily Mongoose allows us to use ES6 Promises instead2. One of the first things I did when creating my API calls was replace all the callbacks with Promises. Here is some code that performs a
find operation on the
Song schema and then returns it as a HTTP response using a promise.
You may have noticed in the last code sample the
res.format() function which returns the song data as an HTTP response. This function also returns either JSON or XML depending on the MIME type specified in the HTTP requests
Accept header. Normally I don't think the amount of development work needed to return both notations would be worth it but Express made it so easy to implement3!
Text searching is how search engines take user input and return a list of results. The full picture of how it works is beyond the scope of this blog (although it would make for a fun discovery some day!) but just know that MongoDB allows for a barebones version of text searching. You can implement a text search by defining an index of type
'text' on a property (most likely you will define it on a string but you could even place it on an array)6. MongoDB will do the rest of the work for you. Weights can also be placed on properties to specify how important a certain field is for a search7.
Remember how I said practically anything you wanted to do in MongoDB can be done in Node.js using Mongoose? You can implement a text search and its necessary indexes as well8! Here are the indexes which are placed on the
Then you can perform the text search using Mongoose:
Now a text search can be executed on the song collection. When a text string is entered, it finds matches on the
best_lyric fields. When I search the word 'faith', it matches a Mariah Carey song with a matching lyric:
I also found a really nice library to add basic auth to my endpoints. Basic auth works by using the HTTP requests
Authorization header to check for a base64 encoded credential string. This credential string before being encoded has the form <username>:<password> followed by the unicode pound sign (U+00A3)9.
All you need to do to implement basic auth in an express app is pass the module a JSON object of usernames and passwords to accept10. This is yet another example of how user made npm modules can make Node.js development easy.
Now all I have to do to run my server is enter one command in bash:
Gulp made my life a lot easier in this project. Combining Gulp which restarted my server on all changes and the WebStorm IDE which automatically saves files, I never had to save a file or restart my server throughout development. I put 100% of my focus on developing!
The code for the prototype is available on my GitHub.