Vagrant Virtual Machine Manager

Vagrant is a virtual machine manager and it makes it easy for creating, running and monitoring of VM’s on your host machines. It’s an open-source and cross-platform (for macOS, Windows, and Linux) software that helps us to manage VMs for VMWare, VirtualBox, AWS (Amazon Web Services). This means you can easily manage different kinds of VMs from one Vagrant configuration which is beautiful!

But what real benefits we can get from it? What are the examples from real life?

First of all, this could heavily increase your development quality! Do you remember how much time did you spend just running projects on your local computer for the first time?

Or you would be much older after doing all of this work for all of your environments like STAGING, TEST, and PRODUCTION. Let’s say you need to set up and run your project in a new environment let’s say: UAT. You will need to prepare all of the project’s dependencies manually. That can react to a one-week period depending on your project’s complexity and size.

But maybe the most beneficial part would be the ability to reproduce production errors and bugs in your development environment. Because it is so easy to run VMs with exactly the same configuration on any computer with Vagrant.

Goodbye “can’t reproduce, runs on my machine” issues.

vagrant

Prerequisites

Vagrant needs a VM manager as a provider. I already told vagrant can use multiple VM managers but for this article, I chose VirtualBox which is an open and free provider presented by default on Vagrant. To download VirtualBox: click here.

Note: Vagrant VM user is vagrant, and its password is vagrant too.

Vagrant setup

Download Vagrant from http://www.vagrantup.com/downloads

Setup Box Path

Note: This is for macOS and Linux only.

Vagrant uses this path to download VMs by default:

~/.vagrant.d/boxes

VMs could take up a lot of space so you might want to change its default path before installing any operating system. To change it just open .bash_profile on your favorite editor (you can use vi instead of nano)

sudo nano ~/.bash_profile

Add this line:

export VAGRANT_HOME="/<TYPE PATH HERE>" New downloads will be under this directory.

Setup operating system

There are lots of Linux distributions provided from Hashi Corp’s Atlas but I chose Ubuntu which I’m already familiar and it is an LTE version! https://atlas.hashicorp.com/ubuntu/boxes/trusty64

Setup as told by the documentation:

vagrant init ubuntu/trusty64
vagrant up

init command will create a file named VagrantFile on the current path. vagrant up the command will use the information on this file.

up command will download and install OS specified on to that VM.

You can see and examine many useful commands with vagrant --help

Connecting to the Box

To see the list of the boxes we type vagrant box list command.

ubuntu/trusty64 (virtualbox, 20151020.0.0)

Now we need to use vagrant ssh, which will connect to the VM written in the VagrantFile.

vagrant@vagrant-ubuntu-trusty-64:~$

My terminal connection is changed to the above which means I’m connected using ssh.

Installing Requirements

These requirements are totally for my node.js example application and will be changed by the project and development/running stack you use.

sudo apt-get -y install git build-essential
curl --silent --location https://deb.nodesource.com/setup_4.x | sudo bash -
sudo apt-get install -y nodejs redis-server
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo "deb http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list
sudo apt-get update
sudo apt-get install -y mongodb-org

Let’s do the Redis configuration: (you can use vi instead of nano) Our application uses Redis with this configuration:

sudo vi /etc/redis/redis.conf

change the line with notify-keyspace-events to this:

notify-keyspace-events "Kgs"

After that restart Redis:

sudo service redis-server restart

Now download and install the application:

git clone https://github.com/dhalsim/chatcat.git cd chatcat git checkout dort npm install sudo npm install grunt-cli -g

We’ll need to write all these commands to a file: provision.sh. Afterward open and change the VagrantFile to this:

Vagrant.configure(2) do |config|
 config.vm.box = "ubuntu/trusty64" config.vm.network "forwarded_port", guest: 3000, host: 8080 config.vm.provision :shell, path: "provision.sh"
end

Exit from ssh with exit command and then type:

vagrant reload --provision

With this line, Vagrant will close, reopen and make installments based on instructions you wrote on provision.sh file.

Vagrant synchronization

It might be hard for you to develop on Vagrant VMs especially with non-graphical terminals. If you’re not used to terminal apps like Vim or Emacs and you want to use your IDE and tools on your host machine, you can do this by enabling Vagrant’s synchronization mode. That could also be used for any data transfer.

I’ll be explaining the usage of synchronization between the Mac OSX host machine and the VirtualBox Ubuntu guest machine. A different combination would need something else and related information can be found on vagrant documentation about synced folders.

Let’s change the VagrantFile:

 Vagrant.configure(2) do |config|
   config.vm.box = "ubuntu/trusty64"
   config.vm.network "forwarded_port", guest: 3000, host: 8080
   config.vm.provision :shell, path: "provision.sh"
   config.vm.synced_folder "~/Projects/chatcat", "/home/vagrant/chatcat"
 end

• We added config.vm.synced_folder command.
• First parameter path, is on the host machine.
• Second parameter path, is on the guest machine

We can check the new options by

vagrant reload
vagrant ssh

Now when we change a file using our favorite editor/IDE from our host machine within the specified path, it will automatically be synchronized by the given guest path.

Port Forwarding

Port forwarding means binding a port number used from the guest machine to the host machine’s any port to be able to use from there.

config.vm.network "forwarded_port", guest: 3000, host: 3000

Now VM’s 3000 port could be used from our computer with 3000 port.

You can try the application running on the guest, by entering this URL to your computer’s browser: http://localhost:3000. Bon appetit :)

Private Network

You can also communicate with your VMs from a private network defined by you. You should change the configuration as follow:

config.vm.network "private_network", ip: "192.168.1.20"

With this method, there is no need for port forwarding, instead localhost you can use this network’s IP address directly and test from here: http://192.168.1.20:3000.

By defining an IP for the guest OS, you can even use your DNS which can be used for clustering.

For VirtualBox use these commands (to make sure VM is closed type vagrant halt):

VBoxManage list vms --> type VM name below 
VBoxManage modifyvm "YOUR_VM'S_NAME" --natdnshostresolver1 on

For local usages, you can set meaningful names on your computer’s hosts file. For Mac OSX change that file using this command (you can use vi instead of nano):

sudo vi /private/etc/hosts

Add 192.168.1.20 nodeapp.com to the bottom of the file and save. After that, we can reach our node.js application from http://nodeapp.com:3000 from the host machine. Furthermore, other VMs can reach that if we made the same configuration on that VMs.

I’ll continue with nodeapp.com below examples.

These changes will cause problems on facebook login and socket processes. Facebook application settings should be changed to use nodeapp.com instead of localhost. These changes will be on config/commons.js on socket_host variable and on test.json file on callbackURL variable. Things to be done:

  1. We need to change application settings from http://developers.facebook.com I suggest you use Test Apps to create a new application and change the Site URL to http://nodeapp.com:3000/.  You need to change test.json with related configuration variables with new AppID and AppSecret given by Facebook.
  2. On config/index.js there is a little issue that I didn’t notice before. We’ll change that to override common settings by the given one:
    module.exports = require('src/lib/utils').extend(commons, environmentConfig);
  3. On test.json add this to override commons.json:
    ... "socket_host": "http://nodeapp.com:3000", ...

Run Application as a Service

Until now, I explained what to be done to install and use our node.js application. Now I’ll explain managing and monitoring it by using a tool named PM2 which is very popular in node.js community. We’ll use it to restart our application on application *crash*es and I also show you how to add our application to the OS as a service of Ubuntu/Linux which automates running application on OS restarts.

Install PM2:

sudo npm install -g pm2

Run this command to install PM2’s command completion tool to help us write many complex commands:

pm2 completion install

You can close and reopen your terminal window for completion to work or run this command:

source ~/.bashrc

NOTE: PM2 has actually lots of handy features for us, but that would be a topic for another article so I’ll pass for now. If you’d like you can search on your own from: http://pm2.keymetrics.io/docs/usage/pm2-doc-single-page/

Before going through service options, we need to make changes on grunt side. Until now our application ran on dev mode. I decided to use test mode for VM. We are still going to use grunt for running Redis and mongo in the same instance but we don’t use watch of grunt or nodemon, instead, we’ll use PM2 for that features.

Spoiler: You can get the code ready for VM by typing git checkout bes but you’d still need to do some of these tasks:

  1. copy dev.json content to a file called test.json.
  2. Type service --status-all to check if Redis and mongo services are active.
  3. Run and test application with NODE_ENV=test pm2 start app.js.
  4. If it runs without any error, kill it with pm2 kill
  5. Add a new file called startup.sh which will provide starting, stopping, etc. of the service. Have a look at variables like NAME, PM2, USER, APP_HOME and change as your needs. For instance, check your user name with whoami (which will be vagrant) or get the path of PM2 executable with which pm2
    #!/bin/bash
      chkconfig: 2345 98 02
     #
      description: our chatcat node app startup script
      processname: chatcat
     #
     ## BEGIN INIT INFO
      Provides:
      Required-Start: $local_fs $remote_fs
      Required-Stop: $local_fs $remote_fs
      Should-Start: $network
      Should-Stop: $network
      Default-Start:        2 3 4 5
      Default-Stop:         0 1 6
      Short-Description: chatcat
      Description: chatcat node app is started
     ## END INIT INFO
     
     NAME=chatcat
     PM2=/usr/bin/pm2
     USER=vagrant
     APP_HOME="/home/vagrant/chatcat"
     
     export NODE_ENV="test"
     
     get_user_shell() {
         local shell=$(getent passwd ${1:-`whoami`} | cut -d: -f7 | sed -e 's/[/fusion_builder_column][fusion_builder_column type="1_1" background_position="left top" background_color="" border_size="" border_color="" border_style="solid" spacing="yes" background_image="" background_repeat="no-repeat" padding="" margin_top="0px" margin_bottom="0px" class="" id="" animation_type="" animation_speed="0.3" animation_direction="left" hide_on_mobile="no" center_content="no" min_height="none"][[:space:]]*$//')
     
         if [[ $shell == *"/sbin/nologin" ]] || [[ $shell == "/bin/false" ]] || [[ -z "$shell" ]];
         then
           shell="/bin/bash"
         fi
     
         echo "$shell"
     }
     
     super() {
         local shell=$(get_user_shell $USER)
         su - $USER -s $shell -c "NODE_ENV=$NODE_ENV $*"
     }
     
     start() {
         echo "Starting $NAME"
         super $PM2 start $APP_HOME/app.js
     }
     
     stop() {
         super $PM2 dump
         super $PM2 delete all
         super $PM2 kill
     }
     
     restart() {
         echo "Restarting $NAME"
         stop
         start
     }
     
     reload() {
         echo "Reloading $NAME"
         super $PM2 reload all
     }
     
     status() {
         echo "Status for $NAME:"
         super $PM2 list
         RETVAL=$?
     }
     
     case "$1" in
         start)
             start
             ;;
         stop)
             stop
             ;;
         status)
             status
             ;;
         restart)
             restart
             ;;
         reload)
             reload
             ;;
         force-reload)
             reload
             ;;
         *)
             echo "Usage: {start|stop|status|restart|reload|force-reload}"
             exit 1
             ;;
     esac
     exit $RETVAL
  6. Copy the file to the path of services with sudo cp startup.sh /etc/init.d/nodeapp.
  7. Give execute the right to the file with sudo chmod a+x /etc/init.d/nodeapp
  8. Run the service with sudo service nodeapp start. You can sudo /etc/init.d/nodeapp start without using service.
  9. Check if the application started with pm2 list.
  10. Stop the service with sudo service nodeapp stop. Check if no application is running with pm2 list and there is no node.js process with ps aux | grep node.
  11. If everything is OK, type this command to register service to start after VM started sudo update-rc.d nodeapp defaults.
    Note: If you encounter any issue trying these, proceed from basic to advanced. For instance, first, check startup.sh is working or not then check the service.
  12.  We also need to make changes to the provision.sh file:
     ...
     git clone https://github.com/dhalsim/chatcat.git
     cd chatcat
     git checkout bes
     npm install
     sudo npm install grunt-cli -g
     sudo npm install -g pm2
     sudo cp startup.sh /etc/init.d/nodeapp
     sudo chmod a+x /etc/init.d/nodeapp
     sudo update-rc.d nodeapp defaults

That’s all for now. If you don’t feel familiar with all these Linux commands and such, there are tons of free sources for Linux from novice to advanced. If you encounter a problem you can’t solve, I’d like to help from the comments.

The next article will be about how to manage logging on different machines using Vagrant to help simulate it. Wait for it :)

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.