This week we started adding “backend” magic to our applications. I’m specifically using the word magic because things got a bit mystical when we started working with certain libraries and frameworks. The instructors at Hack Reactor often say part of being a software engineer is knowing when to “ignore the complexity” so you can avoid rabbit holes and focus on the goal in mind. That was an especially important principal this week.

What we covered this week:

Sprint 1: Servers and Node

In this sprint, we used Node to build a custom back-end. Node is an open-sourced platform that makes it easy to build scalable networking applications because it uses asynchronous, non-blocking I/O. The key here is it offers asynchronous abilities on top of Javascript, a synchronous language, and in practice gives you the ability to write a full stack application in Javascript.

To understand why this is awesome, you should first explore the Javascript event loop. All Javascript code is synchronous and blocking – meaning line 5 of your code won’t run until line 4 is finished, however long that takes. In asynchronous code, your application can run a line of code, do something else while that code runs; you can also attach more code to be run when the first line is finished, this is called a callback.

How to use Node:

1. Installation: do not use brew install node (some weird stuff happens, consider yourself warned), instead use nvm (Node version manager) or follow these instructions
2. Get packages: npm (Node’s package manager) is incredibly rich and very well supported. One of the benefits of using Node is the community, you can find a package for almost anything. In addition to that, npm helps you manage your dependencies so you don’t have to deal with things like virtual machines.
3. Use packages: Node uses the CommonJS pattern for including packages. Simply use the syntax var _ = require(‘underscore’); to include packages in your apps (replace ‘_’ with the variable name you want to use and ‘underscore’ with the package name.
4. Node command line tools: npm installs all packages locally to your app by default. There are many command line tools written in node (ex. nodemon), these must be installed globally using the -g flag.
5. Debugging: type node debug <file> and it will open up a node chrome developer like tool (if you console.log there it will log it in the terminal because its happening on a new server)

Note: Node is currently in a pre-version 1.0 release, despite that it is being utilized by big players like Paypal and Uber – in short, a signal that it is pretty awesome.

More tools! Express is an application on top of Node that makes Node much easier to write. If you google for how to do things in Node, most of the responses you’ll find will pertain to Node with express. From our experiences in class, I definitely recommend using express and not bare Node in most cases. Here is a quick intro to express and all of its magic.

Promises: a solution to “callback hell”
Asynchronous code and callbacks are great and all but then there is this thing called “callback hell”. One function gets called when another is finished, a third is called when that function is finished and so on and so on and so on. Now your code is a huge arrow running off to the right of the page and is typically pretty difficult to follow. Promises to the rescue! A promise represents a value that has not yet been resolved. Promise libraries allow you to chain functions that will be called when that promised value is resolved. Check out Q and BlueBird.

Sprint 2: Server-side Techniques

A server is a system that responds to requests across a computer network. This is a tricky thing to understand so let’s start off with some jargon.

Routing: when you map patterns of URLs that you expect the user to send you to the appropriate routines you want to use to satisfy those requests

Background jobs: a process that runs separate from your primary synchronous code and is set to complete on an auxiliary schedule (i.e. set timers) – this is relative to the programmer’s perspective, a user should never know what is and isn’t a background job

Cron: a way of scheduling/spinning up background processes, check out this awesome tutorial if you’d like to try it

cURL: a library and command-line tool for transferring data using various protocols. I think of it as a way to interact with servers on the command-line, here are some example use cases

Same-origin policy: for security reasons, the browser has introduced restrictions on how you may communicate with domains that are not the one in your URL bar. Restrictions can be strict or loose depending on the type of communication. For example, AJAX is highly restricted and iFrames less so (check out this tutorial for a hack). Two common ways to get around this issue is CORS  and JSONP (find more details on both on the same-origin wiki):

• CORS (cross origin resource sharing) – a new method where (1) the client sends some stuff to the server (in the form of an OPTIONS request), (2) the server sends some stuff back saying it’s okay to send the request, (3) the browser allows the communication and the client send the intended request (usually a GET or POST). This only works if it is integrated on the server side (most websites are CORS compatible now).
• JSONP (JSON with Padding) – JSONP leverages HTML script tags by putting an AJAX call in a padded script tag. Script tags in HTML are immediately invoked. Once invoked, the browser will send the request along with a callback function to the server. (Assuming the server is JSONP compatible) the server will send back the data wrapped in the callback function which is immediately invoked by the browser. Note: only do this with sites you trust because whatever data is sent back will be immediately injected into your code.

JSONP Example:

(in the js file)

var showStockPrice = function(ticker) {
 // do something with the stock price
}

$.ajax({
 dataType: ‘jsonp’
 url: ‘http://other.com/stock.json’,
 data: {ticker: ‘TSLA’},
 success: showStockPrice(response)
})

(in the html file)

<script src=”http://other.com/stock.jsonp?ticker=TSLA&cb=showStockPrice”></script>

This would result in the following code being run:
showStockPrice({price: '$190', mktcap: '23.65B'});

Sprint 3: Databases

A database is a way to store and organize data. In this sprint, the goal was to start thinking about how to organize data and to create persistent data storage (up until this point, any data entered into our apps were gone as soon as we refreshed the page).

Databases are a huge topic, so big that even Hack Reactor cannot condense it (cechk out some MOOCs if you are interested in learning about them). For the purpose of this post, I’ll only cover common database tools used in today’s applications.

Generally there are two main categories SQL and noSQL (or relational and non-relational). noSQL actually means “not only SQL” which can be very confusing.

SQL is a relational database that uses a table structure. This is a type of database with certain conventions, in practice you would actually use an implementation of SQL such as PostgreSQL, MySQL  or SQLite. On a related note, what we know as Javascript is actually the browser’s implementation of ECMA script. ECMA script comes with a base set of features and specs that is offered in every version of Javascript but “Javascript” in Chrome has unique features to Safari

Pros

• Most established: lots of support and tools are available (also if you are starting a company, it’ll be much easier to find people familiar with SQL)
• Structured: SQL has a rigid structure that forces you to do things a certain way, there’s a learning curve to this but following these guidelines will generally keep your data cleaner

Cons

• There is a learning curve
• It is less flexible – you have to plan your data structure ahead of time as future changes will require a lot more effort to implement

Learn more about SQL by reading up at SQLCourse or doing this interactive tutorial.

Mongo is a non-relational database that utilizes document based storage. Mongo is just one example of a noSQL database but it is currently one of the most popular (graph databases like Neo4j are also on the rise).

Pros:

• Flexible
• Quick to startup and easy to learn

Cons:

• It’s relatively new so a lot is still being built and changed, almost every company I know that started on Mongo and eventually decided to migrate to SQL

ORMs (Object-relational mapping) are used in programming languages to interact with a virtual object database. They make it much easier to do CRUD (create, read, update and delete) actions on data. Check out Bookshelf.js if you’re using SQL or Mongoose if you’re using Mongo to start out with.

Feel lost yet? I know I was feeling pretty lost at the end of week 4 (there just isn’t enough time!). There is a ton of information in the post, even more than the past ones. Use this post as a guide of where to go for the best tutorials and explanations and dig into the links for the meat.