Environment Setup for Mobile Web Development

 

xTuple's Mobile Web application is very different from the xTuple ERP desktop client. They have different software architectures, use different tools and development languages, and have different run-time environments. This chapter outlines how to set up a development environment for contributing to the xTuple Mobile Web application development effort.

For the time being these instructions are limited to creating an Ubuntu Linux work environment. This gives us a consistent platform on which to develop, test, and otherwise collaborate. Future versions might expand the description to other environments.

We strongly encourage everybody to work in a virtual machine rather than directly in a workstation host environment. This will isolate your xTuple mobile web development environment from your other work, allowing you to start and stop daemons, reboot your development environment, have completely isolated development environments, test multi-host interactions, etc.

The xTuple Mobile Web development team will try to maintain a baseline working virtual machine (VM) that can be used by anybody who wants to start development. The following instructions are split into three main sections: setting yourself up to work with the master Git repositories on GitHub (very short), creating the baseline VM, and personalizing the VM for use by an individual developer.

This chapter puts all of the source code in ~/src (~ stands for your home directory). There are lots of ways to organize your files. However you choose to do so, pick a scheme and stick with it. Make sure your scheme plans for the future. It should allow you to work on multiple releases simultaneously, switch database versions with little effort, work on different projects (e.g. mobile and desktop), etc., all without forcing any particular directory to get too broad. A foolish consistency is the hobgoblin of little minds... (Emerson) but a smart consistency will save you time and effort.

Version numbers of various components that we use will change over time. This chapter uses XYZ in file and directory names to represent the version number of the applicable software. If directions need to refer to two different components, the first will be XYZ and the second UVW. For example, the section below describing PostgreSQL refers to a directory pgXYZ. For PostgreSQL 9.1.6 this means you should type pg916.

Set Up on GitHub

This section is only required once per person. If you have already set yourself up as a GitHub user and forked the xTuple project code, skip ahead to the next section.

The xTuple mobile web client files are maintained on github.com. If this is your first time working on this project, set yourself up as described in ???. To summarize the process:

  1. Log in to github.com

  2. Navigate to the xTuple project: https://github.com/xtuple/

  3. Fork the client repository:

    1. Click on client

    2. Click on Fork in the upper right-hand corner of the browser window

    3. Click the Fork to your-username button

  4. Return to the xTuple project and repeat the forking process for the other xTuple code repositories.

Create an SSH keypair so GitHub can authenticate your push requests. If you have a public/private encryption keypair you like to use, copy the appropriate files to the .ssh subdirectory of your home directory on this VM. Otherwise generate a new keypair:

xtuple$ ssh-keygen # this isn't extremely secure but it'll do
xtuple$ cat ~/.ssh/id_rsa.pub
ssh-rsa AAA[...about 200 more characters]...M8n8/B xtuple@mobiledevvm

The cat command shows the public key that was just generated. Copy this text, starting with the ssh-rsa at the beginning and ending with the xtuple@mobiledevvm at the end (select and either right-click > Copy or Control-Shift-C in the Linux Terminal window).

In your web browser, navigate to your home page on GitHub. Click on Edit Your Profile. Select SSH Keys from the list on the left. Click Add SSH Key. Give this SSH key a title, such as "xTuple Mobile Dev VM", then paste the public key into the Key field. Finally click the Add key button. GitHub will verify your password just to make sure it's you at the keyboard.

Creating a Baseline Virtual Machine

Create a Virtual Machine

Download and install Oracle VirtualBox. If you already run Linux on your workstation, your distribution may have VirtualBox packages which you can install with your package manager. Otherwise you can get VirtualBox from its website:

  1. Go to www.virtualbox.org

  2. Click Downloads

  3. Click on the link for the VirtualBox platform package for your workstation environment

  4. Install VirtualBox from the downloaded file as app

Now get an Ubuntu 12.x distribution to set up your VM:

  1. Go to www.ubuntu.com/download

  2. Click on Ubuntu Desktop

  3. For "Choose your flavour", select 64 bit unless your workstation has a 32 bit CPU

  4. Click on Get Ubuntu 12.04 LTS

  5. If the website asks for a donation, decide whether to contribute or Not now, take me to the download

  6. Remember where the download process puts the Ubuntu .iso file

Start VirtualBox to create the new VM. You'll need to mount the Ubuntu .iso file as the boot device when the VM first starts up. Along the way, VirtualBox will ask you to set some basic parameters for the VM. Make sure to allocate enough disk space, as this is harder to adjust later than some of the other features of the VM. Take the following steps after VirtualBox starts:

  1. Click the New button in the toolbar

  2. Go through the Create Virtual Machine wizard and use the following settings:

    Name

    Pick a name that will help you distinguish this VM from others like it

    Type

    Linux

    Version

    Ubuntu

    Memory

    1024 MB if your workstation can spare that much

    Create a virtual hard drive

    Select the VDI file type and make it Dynamically allocated

    File location and size

    Leave the default file location but expand the file virtual hard drive to about 20GB

  3. Select the new VM from the main VirtualBox Manager window and click the Start button in the toolbar

  4. VirtualBox will ask you to select a start-up disk. Click the folder-shaped browse icon and select the Ubuntu .iso file you just downloaded, then click the Start button. This will boot the new VM using the .iso file as the boot disk.

  5. Click Install Ubuntu

  6. If the installer asks, select "Erase disk and install Ubuntu", and Continue. The next window should show you the size of the virtual hard disk. If this isn't the ~20 GB set above, Quit and check your VM again. Otherwise Install Now.

    Answer whatever questions the Ubuntu installer asks. Since this isn't expected to be a secure production environment and the VM is intended to be shared, set the username/password pair to xtuple/xtuple and select "Log in automatically". Individual developers can change the password later if necessary.

    Wait until the installer finishes.

  7. When the installer asks you to reboot, do so. The rebooting process may ask you to eject the disk from the CD/DVD drive. Hover over the circular disk icon near the lower-right-hand edge of the VM window. If it reports the drive as Empty, just click on the window and hit the Enter key. If the drive is not empty, right click on the icon, eject the .iso file from the virtual drive, then click in the window and hit the Enter key.

  8. After the VM reboots let the Update Manager upgrade the software that's installed and reboot again if necessary

  9. Install the VirtualBox Guest Additions: From the VirtualBox menu system, select Devices > Install Guest Additions. Inside the Ubuntu VM, click Run in the dialog box that appears. When the process is done, eject the VirtualBox Additions disk image.

  10. Right-click on the gear icon in the upper right corner of the Linux desktop > System Settings > Brightness and Lock. Turn off Lock, turn off the screen saver, and turn off requiring your password when waking from suspend. The defaults are all reasonable for your VM's host; having any of these turned on in your VM, too, is just annoying.

Install Tools, Utilities, and Libraries

Now that you have a virtual machine up and running, it's time to install some basic software. The rest of these instructions assume you are working from a command line even though many of these tasks can also be performed with GUI tools.

First let's open a Terminal window and make it easy to do so again in the future:

  1. Click on the Dash Home icon

  2. At the bottom of the VM's window click on the 3-column icon next to the Home icon

  3. In the Installed list click "See 78 more results"

  4. Scroll down and click the Terminal icon to open a Terminal window

  5. Permanently add Terminal to the Launcher by right-clicking on the icon that just appeared there and select Lock to Launcher

Next you need a text editor. You can use whichever editor you wish for your own work but please make sure that there is some version of vi installed. You will need an editor that runs in a Terminal window at some point and vi is one of these editors that most of the senior developers are familiar with. Without vi it'll be hard for you to get help. Install vi:

root# apt-get install vim

If you want a GUI version of vi, get vim-gnome. After apt-get resolves dependencies, both GUI and command line versions of vi will be installed.

A long-standing tradition in the UNIX world, inherited by GNU/Linux, is that shell prompts end with $ for most users (> if you use csh), and # for superusers. That means that the apt-get command above needs to be run as a superuser. Either sudo apt-get ... to run that one command as a superuser or sudo su to start a subshell for a series of commands. If you choose to start a subshell, either exit or ^D will quit that subshell.

Unfortunately, # is also the shell comment character. Read the commands carefully to distinguish between the two uses of #.

Install Full Development Environment

The entire environment can be setup and installed either using two debian packages, a single startup script or manually.

  • The debian packages are available at asteroidbelt.xtuple.com via FTP
  • The script, built to run on Ubuntu 12.04 LTS, will automatically update and install packages, download and compile dependencies, load an xTuple demo database and install the node server.  The installation process that the script runs through is documented and explained through the entire rest of this article.
  • If you wish to install the environment manually possibly for any reason, this page can be used as a step by step guide.

Once installed, the server may be started from the directory /usr/local/src/xtuple/node-datasource/ by running ./main.js and will listen on https on all interfaces.  You will then be able to login with firefox by pointing it to https://localhost/. The browser will warn about the self signed cert, you must accept the security exception after which you will be redirected to the login page.

Manual Installation

We need to a full C/C++ software development environment to build some of the infrastructure from source and to work with the mobile web client code repositories. Install these basic tools: Git, Subversion, the Package Configuration utilities, C and C++ compilers and supporting tools, and libraries and headers to build programs that use SSL:

root# apt-get install git
root# apt-get install subversion
root# apt-get install build-essential
root# apt-get install libssl-dev

 

Get the Node.js Server

Node.js provides several services for xTuple's mobile web client. Build the Node.js server from a stable version of the source code:

xtuple$ cd ~/src
xtuple$ git clone https://github.com/joyent/node.git
xtuple$ cd node
xtuple$ git checkout v0.8.22
xtuple$ ./configure 
xtuple$ make
xtuple$ sudo make install

 

Get and Build the V8 JavaScript Engine

Another critical piece of xTuple mobile web infrastructure is Google's V8 JavaScript engine. This is integrated into the PostgreSQL database server to allow writing stored procedures in the JavaScript language. To make this work we need to build and install both V8 itself and the PLV8JS integration:

xtuple$ cd ~/src
xtuple$ git clone git://github.com/v8/v8.git
xtuple$ cd v8
xtuple$ make dependencies
xtuple$ make library=shared native
xtuple$ sudo cp out/native/lib.target/libv8.so /usr/lib
xtuple$ cd ..
xtuple$ git clone https://code.google.com/p/plv8js/
xtuple$ cd plv8js
xtuple$ make V8_SRCDIR=../v8 CPLUS_INCLUDE_PATH=../v8/include
xtuple$ sudo su
root# make install
root# exit
If make in the plv8js dir fails with a complaint about pg_config, make sure that /usr/local/pgsql/pgXYZ/bin is in the PATH. That's also why the make install in plv8js requires sudo su first - to avoid path problems. This might have been fixed by tweaks to the PATH variable settings after the pass through this step.

Build and Install PostgreSQL

We do not want to use the generic postgres package that is subject to updating when you least want it to therefore the install script will select postgresql-9.1 from Ubuntu's repos.  You may use the version locked package if your distro supports it.

xtuple$ apt-get install postgresql-9.1

Regardless of the method you use to install postgres, you will need these development packages or their equivalent:

xtuple$ apt-get install postgresql-contrib postgresql-server-dev-9.1

If you are using the packages, you can skip this section on compiling it by hand down to configuring the PostgreSQL runtime environment.

 

To build from source, point your browser to www.postgresql.org and click on Download. Scroll down to Source code and click on file browser. Click on the latest 9.1.x link. Click on either the .tar.bz2 or tar.gz file to download the sources. Save the file if the browser asks; don't open it.

Unpack, configure, and install the PostgreSQL software:

xtuple$ sudo apt-get install libreadline6 libreadline6-dev
xtuple$ cd ~/​src
xtuple$ mkdir postgresql
xtuple$ cd postgresql
xtuple$ tar xzf .../your/download/dir/postgresql-XYZ.tar.gz  # if you downloaded the tar.gz
xtuple$ tar xjf .../your/download/dir/postgresql-XYZ.tar.bz2 # if you downloaded the tar.bz2
xtuple$ cd postgresql-XYZ
xtuple$ ./configure --with-openssl --with-readline --prefix=/usr/local/pgsql/pgXYZ
xtuple$ make
xtuple$ make check
xtuple$ sudo make install
xtuple$ cd contrib/pgcrypto
xtuple$ make
xtuple$ sudo make install

Create a PostgreSQL superuser and data directory for PostgreSQL:

root# adduser postgres
Enter new UNIX password: postgres
Retype new UNIX password: postgres
root# cd /usr/local
root# mkdir pgdata
root# chown postgres pgdata
root# su postgres
postgres$ vi /home/postgres/.bashrc # add the following to the end of the file
PATH=/usr/local/psql/pgXYZ/bin:$PATH
export PATH

postgres$ . ~postgres/.bashrc
postgres$ initdb /usr/local/pgdata/pgXYZ

Configure the PostgreSQL runtime environment

Edit the database server configuration file /etc/postgresql/9.1/main/postgresql.conf. Find the lines that start with the following parameter names, remove the comment character (#) at the beginning, and set the values listed:

Parameter Name Value Notes
listen_addresses '*' The database server should accept connections from all hosts.
custom_variable_classes 'plv8'  
logging_collector on Optional but recommended to help debug database interaction.
log_statement 'all' Also optional but highly recommended to help debug database interaction.

Open up PostgreSQL runtime security — this is a development environment for use on a secure network with demo data, not a production environment. Look at /etc/postgresql/9.1/main/pg_hba.conf and make sure the METHOD column contains trust for the local host and for addresses on the local network.

For a server installed from repos, simply restart the service for the changes to take effect, then restore the database.

xtuple$ sudo service postgresql restart
xtuple$ wget http://downloads.sourceforge.net/project/postbooks/03%20PostBooks-databa...
xtuple$ wget http://downloads.sourceforge.net/project/postbooks/03%20PostBooks-databa...
xtuple$ psql -U postgres -f init.sql
xtuple$ createdb -U postgres -O admin dev
xtuple$ pg_restore -U postgres -d dev postbooks_demo-U.V.W.backup
xtuple$ psql -U postgres dev -c "CREATE EXTENSION plv8"

Now jump to Customising the Baseline Virtual Machine.

For a postgres compiled from source, start the server for the first time, download some files from xTuple's SourceForge site, create some xTuple roles, and create your first database:

postgres$ pg_ctl -D /usr/local/pgdata/pgXYZ start
postgres$ cd
postgres$ mkdir xtupleUVW
postgres$ cd xtupleUVW
postgres$ wget http://downloads.sourceforge.net/project/postbooks/03%20PostBooks-databa...
postgres$ wget http://downloads.sourceforge.net/project/postbooks/03%20PostBooks-databa...
postgres$ psql -U postgres -f init.sql
postgres$ createdb -U postgres -O admin dev
postgres$ pg_restore -U postgres -d dev postbooks_demo-U.V.W.backup
postgres$ psql -U postgres dev -c "CREATE EXTENSION plv8"
postgres$ exit # to quit the subshell and return to being root

Now set the VM to start PostgreSQL on reboot. There are several ways to do this. The easiest is to edit /etc/rc.local:

root# cd /etc
root# vi rc.local # add the following line *before* the exit 0 at the end of the file
sudo -u postgres nohup /usr/local/pgsql/pgXYZ/bin/postgres -D /usr/local/pgdata/pgXYZ

root# exit # to quit the subshell and return to being xtuple

As the last step for getting PostgreSQL up and running, set your own PATH to include the PostgreSQL programs:

xtuple$ vi ~/.bashrc # add the following to the end of the file
PATH=/usr/local/psql/pgXYZ/bin:$PATH
export PATH

xtuple$ . ~/.bashrc

 

Customizing the Baseline Virtual Machine

Set Up the VM for GitHub Interaction

Stop here if you are setting up a VM for distribution. From this point on, all changes to the VM are particular to a single person. If you distribute a VM after passing this point in these instructions, you run the risk that anyone who gets a copy of that VM can impersonate you.

Bring a copy of the main xTuple mobile web client code to your VM. This involves cloning your forks of the database, client, login, node-datasource, node-proxy, node-router, and node-xt repositories, then linking these to the original xTuple repositories:

xtuple$ cd ~/src
xtuple$ git clone git://github.com/your-gituser/xtuple.git
xtuple$ cd xtuple
xtuple$ git remote add XTUPLE git://github.com/xtuple/xtuple.git

The mobile web project uses Git's concept of submodules in some places to import code from other projects. In other places the project uses symbolic links to share code. The general guideline is that xTuple code shared by different repositories is symbolically linked while non-xTuple code is incorporated as submodules. Import the various submodules:

xtuple$ cd ~/src/xtuple
xtuple$ git submodule update --init --recursive
xtuple$ cd node-datasource
xtuple$ npm install
xtuple$ cd node_modules/express
xtuple$ npm install
xtuple$ cd ../../xt
xtuple$ npm install
xtuple$ npm install -g node-gyp
xtuple$ npm install pg
xtuple$ npm rebuild pg

Having imported the submodules, create the ssh keys for the node server to use.

xtuple$ cd ~/src/node-datasource/lib/private

xtuple$ openssl genrsa -des3 -out server.key 1024       # pass: xtuple
xtuple$ openssl rsa -in server.key -out key.pem
xtuple$ openssl req -new -key key.pem -out server.csr
xtuple$ openssl x509 -req -days 365 -in server.csr -signkey key.pem -out server.crt

Create the global database and install the ORMs.  Then load the global database with information about dev.

xtuple$ cd ~/src/xtuple/enyo-client/extensions
xtuple$ ./tools/buildExtensions.sh
xtuple$ cd ../database/source
xtuple$ psql -U postgres -d dev -f init_instance.sql
xtuple$ cd admin/database/source
xtuple$ psql -U postgres -d dev -f "init_script.sql"
xtuple$ cd ~/src/xtuple/node-datasource/database/source
xtuple$ psql -U postgres -d global -f "init_global.sql"

xtuple$ cd ~/src/xtuple/node-datasource/installer
xtuple$ ./installer.js -cli -h localhost -d global -u admin -p 5432 -P admin --path ../database/orm
xtuple$ ./installer.js -cli -h localhost -d dev -u admin -p 5432 -P admin --path ../../enyo-client/database/orm

xtuple$ psql -U postgres global -c "INSERT INTO xt.dbserver (dbserver_name, dbserver_hostname, dbserver_port, dbserver_password, dbserver_username) VALUES ('localhost', 'localhost', 5432, 'admin', 'admin');"
xtuple$ psql -U postgres global -c "INSERT INTO xt.org (org_name, org_dbserver_name, org_licenses, org_active) VALUES ('dev', 'localhost', 10, True);"
xtuple$ psql -U postgres global -c "INSERT INTO xt.usrorg (usrorg_usr_id, usrorg_org_name, usrorg_username) VALUES ('admin', 'dev', 'admin');"

xtuple$ psql -U postgres global -c "INSERT INTO xt.ext (ext_name, ext_descrip, ext_location, ext_priv_name) VALUES ('admin', 'Administration extension', '/public-extensions', 'AccessAdminExtension');"

xtuple$ psql -U postgres global -c "INSERT INTO xt.orgext (orgext_org_name, orgext_ext_id) SELECT 'dev', ext_id from xt.ext WHERE ext_name = 'admin';"
xtuple$ psql -U postgres global -c "UPDATE xt.usr SET usr_password='\$2a\$10\$orE6aDt4lAOkS0eLZPer5OVCYOrVOpiRGhVa3uyueRvW4Mh4BLGeW' WHERE usr_id='admin';"

In the directory ~/src/xtuple/node-datasource edit sample_config.js: change the "bindAddress" parameter from "localhost" to "0.0.0.0". Save this file as config.js.  Now the server may be run by invoking it from this directory.

sudo ./main.js

In your browser of choice, point it to https://localhost/.  It will warn you about an untrusted connection.  This is because you created an unsigned certificate, it is OK to permenantly store this exception for development purposes.  You may see a dialog about being redirected to debug.html; this is expected and you should click OK.  At the login screen, login with admin/somenew and the application should render to completion after a while.

 

 

mead