Lab 3 Details for MPCS 51221

Each lab will consist of a small problem and details of  how to proceed. Each lab is intended to give every student hands-on experience with the core technologies utilized during the course.  A student may concentrate, as a team member, on one technology over another for the final project, but labs are designed to give each and every student exposure to all the technologies that come into play.  You need to submit labs to the TAs for grading--see submission instructions below.  Generally, unless otherwise specified, you will have one week to complete each assigned lab.

See the syllabus for information on grading.  Turning in lab assignments on time is required, without exception, and all late deliveries will be penalized, regardless of cause.  Submit your assignments to the subversion repository according to the directions on the syllabus page.

You may write these solutions in either Java or C++...your choice.

Lab 3   Due: 5:00 pm, Sunday, November 12, 2017

Problem 1:  Playing With Docker and MySQL: 

BACKGROUND:

Like all programming problems, learning a new technology is not an exercise in reading but rather and exercise in typing.  This lab is designed to give you hands-on experience in (a) running a dockerized RMI client program that (b) communicates with an RMI server program in another docker container which then gets sample data from a MySQL database and (b) returns that data to the dockerized RMI client program.  You will generally find the References section below helpful in addition to the required and recommended reading.  When we talk about "Docker", we are talking specifically about the Stable Community Edition of Docker, which is the version we will be using in this class.  The Stable Community Edition provides the basic container engine and built-in orchestration, networking, and security.

Note that this lab will focus on and exercise interprocess communication and accessing MySQL from a remote client via an RPC metaphor (Java RMI).  Please keep in mind that in your actual microservices architecture that you are creating for the final project deliverable, as each microservice has its own datastore, you will likely only be hitting MySQL from a local microservice, which is of course simpler, and not remotely...as shared persistence among microservices is a NO-NO.  Consider also that each microservice will likely have an instance of either MySQL or MongoDB as their datastore, and will be making local calls to that datastore.

WHAT YOU NEED TO DO:

STEP 1:

For this third lab, we are going to code a little microservice that will accept a synchronous call over RMI to a server with a backing MySQL database.  That RMI Server will obtain data from the MySQL database, and return it to an RMI Client running in a separate container.  You will not need to run a script capture on this lab, as you will be submitting your two client and server docker containers.

First, make sure docker is running either on your laptop (Option I from the first lab) or in your VM (Option II from the first lab).

STEP 2:

Get Java RMI working in a Docker Container (CLIENT)

First, get down the Apache Httpd Server version 2.4 container.  Do this by executing the following command (make sure you understand the meaning and significance of -p 8080:8080):

docker run -dit -p 8080:8080 --name Lab3HttpdServer httpd:2.4

Once you have that container down (and running), obtain a bash shell in that container by running docker exec into the container.  Once you are in that httpd container, install some software in the container.  You will update the ubuntu OS, then install the vi editor, the Java 1.7.x JDK, and net-tools, which will allow you to run network-related utilities including ifconfig.

apt-get update
apt-get install vim
apt-get install default-jdk
apt-get install net-tools
apt-get install openssh-client

Create a subdirectory called "/src/lab3/RMI".  Next, download ~mark/pub/51221/src/RMI.tgz and copy it into your container and untar the contents into your RMI subdirectory.  You could also scp it from the Linux cluster directly into your container as well.  Cat the README file and compile the java client and server.  Then execute the server and in a new terminal window execute the client, to make sure everything is running.  Refer again to the README file on how to run the client and server.   You can just use "localhost" for the <server host address> on the client command line.

This container will server as your "Client" container for the synchronous exercise of this lab.  Don't be confused by the name of this container.  Although it is the httpd server and is named Lab3HttpdServer, it will still function as our CLIENT for this lab.


 STEP 3:

Get MySQL working in a Docker Container (SERVER)

Next, in a separate terminal window get down a mysql container by running this command:

docker run --name Lab3MySQLServer -e MYSQL_ROOT_PASSWORD=secret -p 3306:3306 -d mysql

Docker exec a bash shell into your new mysql server and poke around a bit.  Run the following updates in your mysql_server:

 apt-get update
apt-get install vim
apt-get install default-jdk
apt-get install net-tools
apt-get install openssh-client
apt-get install unzip
Go here: https://dev.mysql.com/doc/employee/en/employees-installation.html and follow the instructions to download and load your mysql instance with the sample employees database.  You may find that obtaining the test_db-master.zip file onto your host system, and then copying it into your container will be easiest.  You may also find it easier to obtain the tarball of this sample database off my pub directory ~mark/pub/51221/src/test_db-master.tgz.  It's contents are identical to the zip file refered to on the dev.mysql.com website above.   Once you've decompressed the sample employees database file, run the instructions at the website above to install the sample database into your mysql container (mysql -t < employees.sql).

Create a new subdirectory called "/src/lab3/RMI" in your Lab3MySQLServer container.  Next, download ~mark/pub/51221/src/RMI.tgz and copy it into your container and untar the contents into your RMI subdirectory.  You could also scp it from the Linux cluster directly into your container as well.  Cat the README file and compile the java client and server.  Then execute the server and in a new terminal window execute the client, again making sure everything is running.  Refer again to the README file on how to run the client and server.  Again you can use "localhost" for the <server host address> on the client command line.

Now, let's test out the MySQL installation.  In your Lab3MySQLServer container, execute the following commands:

mysql --user=root --password=secret

mysql> show databases;
mysql> use employees;
mysql> show tables;
mysql> desc employees;
mysql> select * from departments;
 mysql> select count(*) from departments;
mysql> select count(*) from employees;
mysql> select * from employees where first_name like 'D%';

Are you having fun yet?  Assuming that's all working just dandy, you can exit out of the "mysql>" prompt by simply typing "quit". 

This container will serve as your SERVER container for the synchronous exercise of this lab.

STEP 4:

Verify the Java JDK in the MySQL working in the Docker Client Container

Exec into your httpd client container Lab3HttpdServer (if you're not already there).

Now, let's make sure that our Java programs can hit our MySQL server.  Get the MySQL java driver down.  You can get it from my pub server at ~mark/pub/51221/src/mysql-connector-java-5.1.44.tar.  Untar that and store it somewhere like /mysql-driver or /mysql-connector-java-5.1.44 in your Lab3HttpdServer container.  Now, set your CLASSPATH so that it includes this directory, something like:

export CLASSPATH=mysql-connector-java-5.1.44/mysql-connector-java-5.1.44-bin.jar:.

Make sure that you don't forget the final ":." in that line above!

Now, get JdbcTest.java from the cluster at ~mark/pub/51221/src/JdbcTest.java, and copy it into your Lab3MySQLServer container, maybe into the same subdirectory you're using for you RMI code.  Now compile JdbcTest.java.  Make sure whatever terminal you use to execute javac has the CLASSPATH set above.  Edit that file, and change the following lines to match your configuraion:

                        String connectionUrl = "jdbc:mysql://172.17.0.3:3306/employees";
                        String connectionUser = "root";
                        String connectionPassword = "secret";

Find out the IP address of the Server container.  Inside your Lab3HttpdServer container, type ifconfig.  Note the IP address of the eth0 interface in that container (it should be something like "inet addr:172.17.0.2" or something like that). Pay particular attention to the proper IP address of your mysql server container.  It may be 172.17.0.2 or 172.17.0.3 or something else.  You will need to replace the connectionUrl string's IP address above with the correct actual server IP.  Note that the root user and password secret lines should be fine.  Recompile JdbcTest.java and run it.  Hopefully, you will get (after perhaps a warning or two) a bunch of dept_no's and Department Names.  If so, you're in great shape.  Your JdbcTest java test program running on the CLIENT just hit your MySQL instance in your SERVER.

STEP 5:

Verify that your RMIClient program running in your docker client can talk to the RMIServer program running in your docker server. 

Find out the IP address of the Server container.  Inside your Lab3MySQLServer container, type ifconfig.  Note the IP address of the eth0 interface in that container (it should be something like "inet addr:172.17.0.3" or something like that).

Now we need to connect our RMIClient running on our CLIENT (Lab3HttpdServer) to the RMIServer running on our SERVER (Lab3MySQLServer).  To do this, on your Lab3MySQLServer SERVER execute the RMIServer program.

On your Lab3MySQLServer SERVER, execute java RMIServer.   Note the "using address" line, especaially the IP address.

Now, in your CLIENT container ((Lab3HttpdServer), execute your client against your server, using a command similar to this:  "java RMIClient 172.17.0.3 hi".  Of course you would use the correct IP address.  Make sure that your RMIServer prints out the string  you passed in.  Once that's done, you're almost finished.

STEP 6:

Modify your RMIServer and RMIClient to execute a remote database access call and return the result to the remote client. 

On your Lab3MySQLServer SERVER that is running your MySQL instance, use vi (or whatever you've installed as its alternative) to modify the RemoteInterface.java code.  Change the RemoteInterface classes's recieveMessage method so that instead of returning void, it returns an int. 

Then, modify on the CLIENT the source code for RMIClient.java, and change the return value of the receiveMessage() call from void to int, as your new receiveMessage() function will be returning an int instead of a void.

Then, create a function in your RMIServer.java source (very similar to the main() function in JdbcTest class in JdbcTest.java) that when called and passed in a String tablename, hits the (local to the container) mysql database table whose name was passed in, and executes a SELECT COUNT(*) on that table, and returns the numeric result of the number of records in the table returned from the SELECT call.  So, for example, if you pass in the table name "employees" to the client, the remote server would return the int 300024 to the client, because there are 300,024 records in the employees table.  If you were to pass in the table name "departments" to the client (running on CLIENT (Lab3HttpdServer), such as:

# java RMIClient 172.17.0.3 12345 departments

the remote server would return 9, because there are 9 records in the departments table on the server.  So the prototype of the new function might be something like:

    public int execute_it(String tablename)
    {
        //this code will hit the local mysql database table "tablename" and execute a query similar to "select * from departments",
        //and will return the result set's result, in this case, the int "9" because there are 9 records in the departments table...
    }

Then simply have your receiveMessage function that implements RemoteInterface call your new function and return the result, such as:

    public int receiveMessage(String tableName) throws RemoteException
    {
        System.out.println("In RMIServer's receiveMessage method.  Received TableName: " + tableName);
        int y = JdbcTest.execute_it(tableName);  // the return value is the number of rows in tableName
        return y; 
    }


STEP 7:

Run it all. 

Once you have your new changes all executing properly, you're done.  Submit as instructed below.

References:

You may find the following references helpful (in addition to the links from previous labs):

Ubuntu package commands
The docker page for mysql

General Docker Tutorial Links

Docker Cheat Sheet

Submitting:


Create a Bitbucket Repository called Lab 3. Upload your Lab 3 CLIENT and SERVER tarballs (docker export) to this repo.  gzip each tarball before submitting.  Make sure that the user name "johnhb" has access to this repo.  Please include a README text file that contains any instructions for the TAs to assist with grading (especially with instructions on how to run/launch your containers), and design notes are often a useful thing to provide.