Shurn the Awesomer
Decrypting JSON from XE Currency

Decrypting JSON from XE Currency

Written on Thu, 12 January 2017

Disclaimer: What I'm about to show you is against XE Currency's Terms of Use. Use the follow codes at your discretion.

So I have been exploring ways to obtain forex data from external sources. But many websites offer paid services. Me being just a lone developer and not able to afford it, I decided to just scrape of the data from websites. That's where I hit a stumbling block with XE Currency.

An example of the URL to get XE Currency historic rates is http://www.xe.com/currencycharts/currates.php?from=USD&to=SGD&amount=1&_=1484224006201. You will see that this isn't JSON at all. Something is amissed. It is using this URL, the javascript interprets it as JSON, yet this looks nothing like JSON.

So after much delving on this matter, I realised, not to my surprise, that it is encrypted. Interestingly, they do not encrypt the entire JSON, only every 10th character of the JSON string. The rest of the string is just obfuscated. Both the encryption and the obfuscation looks very similar to caesar cipher.

So if you are going to scrape the data off their website, I'll show you the codes I used. I doubt there will be any legal issues for me posting this, since their entire decryption algorithm is publicly accessible. I merely took their algorithm and implement it in PHP.

//Once you get the encrypted JSON, put it here.
$encryptedJSON = "ENCRYPTEDJSONHERE";

//Trim any whitespaces
$encryptedJSON = trim($encryptedJSON);

//The decryption key is the last 4 characters of the encrypted JSON
$hiddenKey = substr($encryptedJSON, strlen($encryptedJSON) - 4);

$decryptedKey = ord(substr($hiddenKey, 0)) + ord(substr($hiddenKey, 1)) + ord(substr($hiddenKey, 2)) + ord(substr($hiddenKey, 3));
$decryptedKey = (strlen($encryptedJSON) - 10) % $decryptedKey;
$decryptedKey = ($decryptedKey > strlen($encryptedJSON) - 10 - 4) ? (strlen($encryptedJSON) - 10 - 4) : $decryptedKey;

//The actual decryption key is hidden in the middle of the JSON
$decryptedKey2 = substr($encryptedJSON, $decryptedKey, 10);

//Remove the encryption key from the JSON string
$encryptedJSON = substr($encryptedJSON, 0, $decryptedKey).substr($encryptedJSON, $decryptedKey + 10);

//Decode URI
$encryptedJSON = urldecode($encryptedJSON);

//Character shift process, doesn't involve any key
$stringList = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
$cursor = 0;
$shiftedJSON = "";

//Clear Unwanted characters
$encryptedJSON = preg_replace("/[^A-Za-z0-9\+\/\=]/m", "", $encryptedJSON);

do {
$char1 = strpos($stringList, substr($encryptedJSON, $cursor++, 1));
$char2 = strpos($stringList, substr($encryptedJSON, $cursor++, 1));
$char3 = strpos($stringList, substr($encryptedJSON, $cursor++, 1));
$char4 = strpos($stringList, substr($encryptedJSON, $cursor++, 1));

$code1 = ($char1 << 2) | ($char2 >> 4);
$code2 = (($char2 & 15) << 4) | ($char3 >> 2);
$code3 = (($char3 & 3) << 6) | $char4;

$shiftedJSON .= chr($code1);
if ($char3 != 64) {
$shiftedJSON .= chr($code2);
}
if ($char4 != 64) {
$shiftedJSON .= chr($code3);
}
} while ($cursor < strlen($encryptedJSON));
$encryptedJSON = urldecode($shiftedJSON);

//Decrypt every 10th character
$counter = 0;
$decryptedJSON = "";
for ($counter10th = 0; $counter10th < strlen($encryptedJSON); $counter10th+=10) {
$encryptedChar = $encryptedJSON[$counter10th];
$shiftKey = $decryptedKey2[($counter % strlen($decryptedKey2) - 1) < 0 ? (strlen($decryptedKey2) + ($counter % strlen($decryptedKey2) - 1)) : ($counter % strlen($decryptedKey2) - 1)];
$encryptedChar = chr(ord($encryptedChar) - ord($shiftKey));
$decryptedJSON .= $encryptedChar.substr($encryptedJSON, $counter10th + 1, 9);
$counter++;
}

//There you go! The decrypted JSON.
echo json_decode($decryptedJSON);

A reminder to read my disclaimer!

Controlling Docker Containers with PHP

Controlling Docker Containers with PHP

Written on Fri, 6 January 2017

After searching through the internet to find out how to manage docker containers through PHP, I finally figured out how to do it. I assume you are using Ubuntu 16.04 and the docker you are using is the latest to date.

Add www-data to docker


According to the documentation, the docker daemon always runs as the root user. Since Docker version 0.5.2, the docker daemon binds to a Unix socket instead of a TCP port. By default that Unix socket is owned by the user root, and so, by default, you can access it with sudo.

Starting in version 0.5.3, if you (or your Docker installer) create a Unix group called docker and add users to it, then the docker daemon will make the ownership of the Unix socket read/writable by the docker group when the daemon starts. The docker daemon must always run as the root user, but if you run the docker client as a user in the docker group then you don't need to add sudo to all the client commands.

So we are going to add www-data to docker so that it can run docker commands. First, let's create the group if it doesn't exist. Skip this command if the group exists.

sudo groupadd docker

Add www-data to docker group.

sudo gpasswd -a www-data docker

Reboot the computer for all the permissions to take effect.

sudo reboot

Writing docker commands


In PHP, you will need to specify the commands in this syntax:

shell_exec("RET=`docker command `;echo $RET');

For example, if you want to run hello world, you write this:

shell_exec('RET=`docker run hello-world`;echo $RET');

Build yourself a Comprehensive Business Suite with Odoo

Build yourself a Comprehensive Business Suite with Odoo

Written on Fri, 30 December 2016

Odoo is an impressive Business Suite that contains so many enterprise management tools. The best part of Odoo is that most of the features are free. There are only a few paid features that would be very useful if you are managing more than 50 employees in your company. Of course, it is up to your use case to see if the enterprise features are useful. In this tutorial, I'll show you how to set up your own Odoo 10 Community Edition, which you can freely use if you are running a small company with very little employees.


Setup Odoo 10


Installing Odoo 10 on Ubuntu 16.04 is easy. Much of this tutorial is taken from the documentation. Feel free to review the documentation when in doubt.

Keep Ubuntu updated with the latest packages

apt-get update && apt-get -y dist-upgrade && reboot


Reboot your server to ensure everything is loaded with the latest packages.

Let's start with adding the repository for Odoo 10, then installing it.

wget -O - https://nightly.odoo.com/odoo.key | apt-key add -
echo "deb http://nightly.odoo.com/10.0/nightly/deb/ ./" >> /etc/apt/sources.list
apt-get update && apt-get install -y odoo

The configuration can be found in /etc/odoo/odoo.conf

nano /etc/odoo/odoo.conf


If you wish to set things to default, just leave the file as it is.


Running Odoo on port 80


If you're like me, I don't like to type the port number in the URL. And my server is running just 1 application. So why not use port 80?

nano /etc/rc.local


Add the following before exit 0 command.

iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8069
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8070
iptables-save


Then reboot the server

reboot

Browse to the server with your preferred browser, such as http://localhost if you have changed port, or http://localhost:8069 if you left it as default. From here on, it's straight forward installing modules and applications for your company needs. More information as Odoo's user guide.



If you're a developer looking to develop for Odoo 10, I recommend reading Odoo 10 Development Essentials. Companies who hire Odoo developers are usually looking to streamline their business to be more efficient. Many big companies will want to do that to reduce operational cost.

Creating a Hybrid Data Centre with OpenVPN

Creating a Hybrid Data Centre with OpenVPN

Written on Mon, 19 December 2016

Setting up a hybrid data centre comprising of on-premise data centre and cloud data centre isn't a difficult task. You do, however, need some knowledge of networking before you proceed with this tutorial. In this tutorial, I'll be focusing on AWS as the cloud provider.

Having hybrid data centre is like having best of both worlds, but do take note that you will also have the worst of both worlds.

Advantages
  • Data compliance - If you have a compliance to meet, having data stored on-premise will certainly help. Especially if it's critical data and infrastructure.
  • Cost Savings On-premise - This is only true in certain use-cases. A typical server rack can cost USD10 000 and potentially beyond. But if properly virtualised and containerised, you could run more than 100 virtual machines. Let's take a m4.large instance from AWS running Ubuntu. It will take you 10 years to your Return On Investment (ROI) for having on-premise server. But if you're running 2 m4.large, that's going to be a 5 year ROI for having on-premise server. The math goes on. After all, a typical server rack last for 5 years.
  • Cost Savings Cloud - This is also only true in certain use-cases. Let's say you only need a server for a short period of time, you could just spin up an AWS instance for just that period of time and pay for that time. This is only good if you don't have enough on-premise resource and don't intend to procure servers for just 1 cause. This is a good strategy where you launch enough cloud resources until procurement of a physical server is justified. That way, you don't need to plan ahead, and just procure equipments whenever cost is justified.
  • Agility - Having a cloud provider gives you agility to have a server instance within seconds. Procurement of server racks take days, in some cases weeks and months, if you have multiple company management approvals to obtain.
  • Ease of Management - You don't need to have expertise to manage cloud data centre, apart from your on-premise data centre. Those are taken care of by the cloud provider.
  • Innovation Leverage - Whenever the cloud provider innovates their product, you can take advantage of the innovation as well. AWS is innovating at an average of 3 innovation per day in 2016. This rate of innovation is seriously unheard of.
  • Reliability - A data centre like AWS typically have more backup systems in place than a typical SME. For instance, they would have multiple backup generator that an ensure service continuity in the event of prolong blackouts while you would most likely have 1 backup battery that only last 20mins. AWS also have Availability Zones(AZ) which has independent internet connections and power supply. It's simply an isolated data centre per zone. If you build your application to run across multiple AZ, you will have greater reliability. Compared to your on-premise data centre, which is likely just a physical location, having multiple data centre will increase reliability.
Disadvantages
  • HR Cost - You still need manpower to manage your on-premise data centre. If you're managing many physical servers, then it would make financial sense. Otherwise, you are better off running the your applications in the cloud where the expertise are already in place to maintain the physical servers.
  • Configuration Nightmare - If the network is not properly set up, you are going to have a nightmare trying to get your applications to communicate. In some cases, this is going to cost your business.

Preparing AWS


Let's prepare the cloud data centre to a specific configuration. If you need a different configuration, you may deviate from this tutorial. But if you're just starting out, I recommend that you follow through this tutorial before setting up your own network configuration.

The network configuration is done in such a way that there is a public subnet and a private subnet per Availability Zone (AZ). I assume that you do not want some applications to have public address, therefore, a private subnet would be suitable for such applications. I'm using Northern Virginia Region with 4 AZ, so that means I will have a total of 8 subnets. I'm only entitled to Zone A/C/D/E, so my tutorial will feature these zones. The AZs available to you might be different, so adjust the commands accordingly as you follow through. You will also need different subnets if you are using AWS features like Aurora, which requires at least 3 different AZ for it to work.

Create the VPC


I've created a VPC named MyAwesomeVPC with the network of 10.0.0.0/16.

Create the Subnets


Let's create the 8 subnets to use. I usually name my subnets properly, such as [Name-Zone-PublicOrPrivate]. If you ever scale your data centre, a good name will help ease administration.

Here's my list of subnets created:

  • Subnet0-ZoneA-Public
    • 10.0.0.0/24
  • Subnet1-ZoneA-Private
    • 10.0.1.0/24
  • Subnet2-ZoneC-Public
    • 10.0.2.0/24
  • Subnet3-ZoneC-Private
    • 10.0.3.0/24
  • Subnet4-ZoneD-Public
    • 10.0.4.0/24
  • Subnet5-ZoneD-Private
    • 10.0.5.0/24
  • Subnet6-ZoneE-Public
    • 10.0.6.0/24
  • Subnet7-ZoneE-Private
    • 10.0.7.0/24

Now that we are done creating all the subnets in their respective AZ, so proceed.

Creating Internet Gateway


The internet gate allows instances created in the subnets to have internet access. Let's create an internet gateway and assign it to our VPC.

Creating Route Tables


Route tables allow proper network routing within the VPC. We will need 2 routing tables, one that has internet access and the other that doesn't.

In the first routing table, we make a public one named MyAwesomeVPC-Public and associate the subnets to it:

  • Subnet0-ZoneA-Public
  • Subnet2-ZoneC-Public
  • Subnet4-ZoneD-Public
  • Subnet6-ZoneE-Public


Now, under routes, let's add the internet gateway we created before. The destination is 0.0.0.0/0, which implies all IP. Internet Gateway ID starts with igw-.

The second routing table will be a private on named MyAwesomeVPC-Private. These are the subnets to associate it with:

  • Subnet1-ZoneA-Public
  • Subnet3-ZoneC-Public
  • Subnet5-ZoneD-Public
  • Subnet7-ZoneE-Public


We don't add the internet gateway because these subnets will not have internet.

We are done for now. We will be back here after we set up a VPN server.

Setting up OpenVPN server


We are going to set up an OpenVPN Server on Ubuntu 16.04. In most cases, a t2.nano server is enough. Upgrade the instance type if you experience hitting your credit limits regularly. You will also just be installing the server software. There is no storage of data for the traffic. The default 8GB HDD storage is more than sufficient.

The ports required for OpenVPN Server is only 22 for SSH, and TCP/UDP port 1194. If you are certain that your VPN clients will only use either TCP or UDP, you can specify only to allow one of it accordingly. For this tutorial, let's enable both. It's not a major security hole at this point in time anyway.

Don't forget to assign an Elastic IP to ensure your IP doesn't change for any reason.

All Linux/Ubuntu commands are running with root account. If you're not running root account, be sure to add sudo before every command.

sudo su

Update Ubuntu softwares to the latest version and reboot the instance.

apt-get update && apt-get -y dist-upgrade && reboot

Install OpenVPN server and it's related softwares.

apt-get install openvpn easy-rsa

Certificate Authority


Set up a Certificate Authority(CA) directory.

make-cadir ~/openvpn-ca

Configure the CA values

cd ~/openvpn-ca
nano vars

Search for the following and edit the values accordingly. You could leave it as default.

export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"
export KEY_OU="MyOrganizationalUnit"


Do not close this file yet. Search for the next configuration:

export KEY_NAME="EasyRSA"


Change it to:

export KEY_NAME="server"


Now save and close the file

Source the vars file:

source vars


You should see this once completed:

NOTE: If you run ./clean-all, I will be doing a rm -rf on /root/openvpn-ca/keys

Time to clean up:

./clean-all

Now build root CA:

./build-ca


Just hit enter as it prompts you for input. The values should be the same as what you typed in the configuration.

Now generate Server Certificates, Key, and Encryption files.

./build-key-server server


You can leave most of the input as default. When you are asked to sign the certificate and commit, choose y.

Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y

Let's generate a strong Diffie-Hellman keys to use during key exchange by typing:

./build-dh


This may take some time, so grab a cup of tea.

Generate a HMAC signature.

openvpn --genkey --secret keys/ta.key

Generating Clients


Now let's generate client keys. You can repeat this step as many as you need for the number of clients you have. I recommend generating a few more than what you have. Each time you generate new clients, you need to restart OpenVPN server for the new clients to take effect, which can be disruptive to existing connections. If you are generating more than 1 client, be sure to use unique names.

cd ~/openvpn-ca
source vars
./build-key client1


You can leave the inputs as default. Password is not required so that you can have automated access. Sign the certificate and commit with 'y'.

Configuring OpenVPN


Now we configure OpenVPN with the files we have generated. Copy them to /etc/openvpn

cd ~/openvpn-ca/keys
cp ca.crt ca.key server.crt server.key ta.key dh2048.pem /etc/openvpn

Get the sample OpenVPN configuration file into configuration directory so that we can use it as a basis for our setup:

gunzip -c /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz | sudo tee /etc/openvpn/server.conf

Configure the openvpn server file we have just unzipped:

nano /etc/openvpn/server.conf


I personally prefer TCP instead of UDP, so find and set the following:

proto tcp


For administrative purposes, I prefer to keep OpenVPN client subnet number equal to host subnet +1. Since our VPC is running on 10.0.0.0/16, I want my clients to be on subnet 10.1.0.0/16. You can leave this as default if you prefer.

server 10.1.0.0 255.255.0.0


In my set up, I allow the clients to communicate with each other. If your set up disallows it, leave this configuration commented out.

client-to-client


Let's ensure all traffic can be routed to 10.0.0.0/8. Otherwise, a connection without ability to access other host in the VPC is not very productive in our use case.

push "route 10.0.0.0 255.0.0.0"


Uncomment the following, and add the additional configuration if it doesn't exists:

tls-auth ta.key 0 # This file is secret
key-direction 0


Uncomment the following encryption type:

cipher AES-128-CBC


Add the following just below the encryption you just uncommented:

auth SHA256


Now uncomment the following:

user nobody
group nogroup


You are done with OpenVPN configuration. Save the file and close it.

Networking Configuration


Let's adjust some networking configurations to ensure network traffic is properly forwarded.

nano /etc/sysctl.conf


Uncomment the following:

net.ipv4.ip_forward=1


Save the file and do the following command to read the new settings:

sysctl -p

Firewall and NAT configurations


I assume that your public network interface is eth0. You will need to edit your /etc/ufw/before.rules file to add in relevant configurations.

nano /etc/ufw/before.rules


Now add the follow codes at the top of the file. It is important that these codes are added before the rest of the existing rules.

# START OPENVPN RULES
# NAT table rules
*nat
:POSTROUTING ACCEPT [0:0]
# Allow traffic from OpenVPN client to eth0
-A POSTROUTING -s 10.1.0.0/8 -o eth0 -j MASQUERADE
COMMIT
# END OPENVPN RULES

Save the file and close. Now we edit /etc/default/ufw.

nano /etc/default/ufw


Find this code:

DEFAULT_FORWARD_POLICY="DROP"


Change it to:

DEFAULT_FORWARD_POLICY="ACCEPT"

Save the file and close. We enable the relevant ports open in the firewall.

ufw allow 1194/udp
ufw allow 1194/tcp
ufw allow OpenSSH

Let's restart the firewall.

ufw disable && ufw enable

Start OpenVPN Server


Let's start OpenVPN server.

systemctl start openvpn@server

You can check the status of the server with:

systemctl status openvpn@server


You can also check the IP of the tunnel server with:

ip addr show tun0

If you're satisfied with all the configuration you've done, proceed to enable OpenVPN to autostart during boot.

systemctl enable openvpn@server

Configure VPC Routing


Now that we have created the OpenVPN server, we must update VPC to route traffic to 10.1.0.0/16 properly. You must do this for MyAwesomeVPC-public and MyAwesomeVPC-private. You will need to get the Network ID of the EC2 hosts. Network ID starts with eni-.

Then under your VPC Route Table, set 10.1.0.0/16 to the Network ID on both tables, public and private.

Now, your VPC EC2 instances will be able to communicate with your clients.

Configure OpenVPN clients


So now that we have set up the server, let's make sure the client works too. First of all, we need to get the client files we generated from the server. If you have followed this tutorial closely, the files you need should be located at:

  • ~/openvpn-ca/keys/client1.key
  • ~/openvpn-ca/keys/ta.key
  • ~/openvpn-ca/keys/client1.crt
  • ~/openvpn-ca/keys/ca.crt


Download them with any softwares that allows you to download from the server. Personally I use filezilla to download via SFTP.

I'm using Windows, so if you're using other OS, adjust accordingly. There are guides for OSX, Ubuntu, Android, and iOS.

Download and install Windows Client


Download OpenVPN client. I recommend getting 64bit, because it's the future. And my tutorial will be based on 64bit. Use the latest stable version, unless you know what you're doing.

The installation will be very straight forward. Leave the installation folder as default, which is C:\Program Files\OpenVPN. Agree to the terms and conditions, which is just simply, if anything screws up, don't fault OpenVPN. But so far, OpenVPN is running smooth for me and many others on the planet. So you should be fine too.

Client Configurations


Your OpenVPN should be installed in C:\Program Files\OpenVPN. Go to C:\Program Files\OpenVPN\config and create a folder called keys. Remember the 4 files you downloaded from OpenVPN server? Move client1.key, ta.key, client1.crt, and ca.crt to C:\Program Files\OpenVPN\config\keys.

Now go back to C:\Program Files\OpenVPN\config and create a file called MyAwesomeVPN.ovpn. Paste the following, but replace the server IP address with yours:

client
resolv-retry 20
keepalive 10 60
nobind
mute-replay-warnings
ns-cert-type server
comp-lzo
max-routes 500
verb 1
persist-key
persist-tun
dev tun
proto tcp
port 1194
cipher AES-128-CBC
auth SHA256
key-direction 1
cert keys/client1.crt
key keys/client1.key
ca keys/ca.crt
tls-auth keys/ta.key
remote 12.34.56.78 1194 # This is your server IP address and port
#redirect-gateway def1 # Only enable this if you want to route all traffic through OpenVPN server

Now in your OpenVPN taskbar icon, when you right-click on it, you will see that you have MyAwesomeVPN to connect to. Connect and make sure it works.

That's it! Now you have a data centre on the cloud that works like your own data centre. You access your server instances like normal Local Area Network(LAN) IP address. But wait, didn't we talk about hybrid data centre where you connect your on-premise data centre to your cloud data centre? Fret on, the steps for setting that up is very similar to how you do on Windows Client. In most cases, you need an extra hardware. Proceed to further reading for more information.

Further Reading

OpenWRT


OpenWRT has the advantage that it will probably work on your existing router, or maybe an old router you have lying around. Check their list of supported hardware before proceeding. An important word of caution, if done improperly, it might brick your router.

Check out Linksys WRT3200ACM. It is the best router available in the market that is made specifically for OpenWRT and other open source router firmware. Featuring MUMIMO for simultaneous wifi connections, dual-band 2.4GHZ and 5GHZ for wifi connection, 1.8GHZ processor for heavy encrypting/decrypting of OpenVPN packets, and 512MB of RAM for you to run other applications such as TOR, Anti-virus scan, etc...

If you're not willing to go through the hassle of OpenWRT installation, you can get preinstalled routers such as the GL-AR150 from GL Technologies. Very convenient and small, makes it easy for installation right next to your network point. It is only powered by USB, so if your router has a USB port with power, it will just work from there. Just note that it only has 64MB of RAM and a 400MHZ processor, which means it is only capable of limited OpenVPN encrypting and decrypting. If you have expecting heavy traffic usage between on-premise and cloud data centre, you might want to look at Linksys WRT3200ACM or even a dedicated CPU running Untangle or pfSense.

Untangle


Untangle has an impressive number of network applications designed to do more than just OpenVPN. It has web content filtering, anti-spam, anti-virus, anti-phishing, anti-spyware, intrusion prevention, firewall, bandwidth management, web cache, and many others. The management console for OpenVPN allows you to easily manage OpenVPN clients. OpenVPN in Untangle acts as both a client and a server at the same time. While you connect your Untangle to your cloud data centre, you can also have other clients connect to Untangle. This is by far, the simplest free and open source OpenVPN solution I've ever tried.

If you have an old hardware lying around, you probably can use it already. The only thing your CPU might be missing is an additional network port. If you computer is manufactured in the last 5 years, it will probably run well with enough CPU power and RAM to do all your OpenVPN needs and more. Check out their hardware requirements for more information.

Alternatively, you could just get pre-installed Untangle firewall hardware from JLTCtech. It will easily support about 25 devices in your network with modest OpenVPN traffic usage.

PfSense


PfSense is a firewall suite with very low hardware requirements. It is capable of supporting very old hardware while still maintaing many features available. Like Untangle, it supports an impressive number of network applications. However, like Untangle, OpenVPN still requires good computational power to encrypt/decrypt packets. It's just how encryption and decryption works.

If you have an old hardware lying around, you probably can use it already. The only thing your CPU might be missing is an additional network port. As pfSense has very very low requirements, any computer manufactured in the last 10 years will probably run well with enough CPU power and RAM for your OpenVPN needs and more. Check out their hardware requirements for more information.

Alternatively, you could just get pre-installed pfSense firewall hardware. This device will easily support about 50 devices with modest OpenVPN traffic usage.

Conclusion


OpenVPN is a very powerful tool you can use to establish remote connections to work virtually in the office. While setup may be a little complicated, it is usually a one time event. Add to the fact there there are applications like Untangle that drastically simplfies your setup, this is easily the best VPN solution around.

Creating a Classroom Suite with Moodle and BigBlueButton

Creating a Classroom Suite with Moodle and BigBlueButton

Written on Sat, 10 December 2016

Moodle is quite a feature rich Learning Management System. If you're like me, wishes to install moodle on your own server, here's a guide to show you how. Afterwards, I'll even show you how to install BigBlueButton to get it to work with Moodle. Combining LMS with a web conferencing tool and you get a full online classroom suite. What more power can you get from that?

In this tutorial, I'll be using Ubuntu 16.04 for Moodle and Ubuntu 14.04 for BigBlueButton. It is most unfortunate that the latest stable version of BigBlueButton won't work on the latest Ubuntu LTS. Although BigBlueButton 1.1 will support 16.04, there is no official news on its release date.

Installing Moodle


As usual, log into root account for ease of command line typing. If not, you will have sudo every other command.

sudo su

Update your Ubuntu Machine to the latest version.

apt-get update
apt-get -y dist-upgrade

Install Prerequisites

apt-get -y install postgresql postgresql-contrib apache2 php libapache2-mod-php php7.0-pgsql php7.0-xml php7.0-curl php7.0-zip php7.0-gd php7.0-mbstring php7.0-xmlrpc php7.0-soap php7.0-intl

Installing Postgres


This step is optional if you are using RDS. I highly recommend using RDS for database as you can take advantage of RDS features.

Set Password by logging in as user postgres, and going into postgres terminal

su - postgres
psql
\password postgres


Prepare a database and a user for moodle

CREATE USER moodleuser WITH PASSWORD 'yourpassword';
CREATE DATABASE moodle WITH OWNER moodleuser;


List the database to ensure you have created it properly.

\l


Now exit from postgres terminal

\q


And log out from postgres user

exit

Installing Moodle


Git clone the moodle application

cd /var/www
rm html/index.html
git clone --depth=1 -b MOODLE_32_STABLE git://git.moodle.org/moodle.git html
chmod -R 0775 html
find html -type f -exec chmod 0664 {} \;

You must create a secure moodle data directory. This directory will be used to store data related to your Moodle isntallation. It is not meant for public internet. If your computer is a shared terminal, please consult your administrator to see what permissions is most suitable.

mkdir moodledata
chmod 775 moodledata

Now browse to your Moodle installation on your web browser. You would be greeted to select your language. Proceed with your preferred language. I prefer English and the tutorial will be in English.

You will then need to confirm that the paths on your server is accurate. Most importantly, the moodle data directory. If you have followed the tutorial, the path will be /var/www/moodledata.

Choose your database. If you have followed the tutorial so far, you would choose PostgreSQL.

Enter the configuration for your PostgreSQL. If you have followed the tutorial so far, your host will be localhost. If you have used RDS instead, you will have to use the details provided by RDS.

Finally, the general configurations for Moodle.

When you are done with the installation, Moodle may not be able to write to the directory, so you might need to do it yourself. Create the file and paste the codes provided at the end of the installation.

nano html/config.php

Setting up cron job


Setting up cron job for moodle is important for moodle to function normally.

crontab -u www-data -e
*/1 * * * * /usr/bin/php /var/www/html/admin/cli/cron.php >/dev/null


Now cron job will run every 1 minute.

Setting up clamav


You should never trust anybody on the internet. That's why you need to install clamav in the server and scan every file that comes along.

apt-get install clamav


To update virus definition, run this:

freshclam


You can also use cron job to update virus definition automatically.

crontab -e


Add the follow for hourly updates. Change the rate of update if this is too much for you.

0 * * * * freshclam


In Moodle, enable the antivirus plugin under Site Administration > Plugins > Antivirus Plugin > Manage antivirus plugins. Then set up the configurations under ClamAV antivirus page.

That's it for Moodle. If you are interested in having online classrooms, proceed to the next part of the tutorial.

Installing BigBlueButton 1.0


So far I have not been able to get a working BigBlueButton installation on any EC2 instances on anything less than 4GB ram. So the minimum requirement is T2.medium instance.

Update Ubuntu libraries to the latest and reboot.

apt-get update && apt-get -y dist-upgrade && reboot

In order for your ubuntu to be supported till April 2019, you need to update the kernel to xenial release.

apt-get install --install-recommends linux-generic-lts-xenial && reboot

Now, you may wish to remove any unused package remaining in your Ubuntu.

apt-get autoremove

Ensure that multiverse in enabled in your source list. Otherwise, enable it.

grep "multiverse" /etc/apt/sources.list


To enable it, use this command:

echo "deb http://us.archive.ubuntu.com/ubuntu/ trusty multiverse" | sudo tee -a /etc/apt/sources.list


Do another update and dist-upgrade if you have added the multiverse.

apt-get update && apt-get -y dist-upgrade && reboot

Installing the latest libreoffice


You cannot use the default libreoffice in the repository as it is not stable. You will need to use libreoffice 4.4 and above. As of writing, the latest version is 5.2.

apt-get install software-properties-common
add-apt-repository ppa:libreoffice/libreoffice-5-2


You will also need ECDSA keys in the audio connection between the browser and FreeSWITCH server.

add-apt-repository -y ppa:ondrej/php

FFMPEG installation


For reasons beyond the scope of this tutorial, FFMPEG is not available in Ubuntu 14.04, so you have to compile it on your own.
Create a file called install-ffmpeg.sh and add the following codes into the file.

sudo apt-get install build-essential git-core checkinstall yasm texi2html libvorbis-dev libx11-dev libvpx-dev libxfixes-dev zlib1g-dev pkg-config netcat libncurses5-dev

FFMPEG_VERSION=2.3.3

cd /usr/local/src
if [ ! -d "/usr/local/src/ffmpeg-${FFMPEG_VERSION}" ]; then
sudo wget "http://ffmpeg.org/releases/ffmpeg-${FFMPEG_VERSION}.tar.bz2"
sudo tar -xjf "ffmpeg-${FFMPEG_VERSION}.tar.bz2"
fi

cd "ffmpeg-${FFMPEG_VERSION}"
sudo ./configure --enable-version3 --enable-postproc --enable-libvorbis --enable-libvpx
sudo make
sudo checkinstall --pkgname=ffmpeg --pkgversion="5:${FFMPEG_VERSION}" --backup=no --deldoc=yes --default

Now run the compilation script.

chmod +x install-ffmpeg.sh
./install-ffmpeg.sh

Run the following to ensure the FFMPEG is installed with version 2.3.3

ffmpeg -version

Actual BigBlueButton installation


It's time to install BigBlueButton by adding the package repository.

wget http://ubuntu.bigbluebutton.org/bigbluebutton.asc -O- | sudo apt-key add -
echo "deb http://ubuntu.bigbluebutton.org/trusty-1-0/ bigbluebutton-trusty main" | sudo tee /etc/apt/sources.list.d/bigbluebutton.list
sudo apt-get update
apt-get install bigbluebutton


This will take some time, so grab a coffee and relax while the installation progress.

Optional API demo


If you would like to, you can install a demo API to test that your installation is working.

apt-get install bbb-demo


This installation demo is exactly the same as the Official Demo Server.
Once you are done testing, you should remove it.

apt-get purge bbb-demo

Optional hostname setup


If you are hosting your set up on the internet, you will likely need to use your hostname instead of IP address. You will need to run the following command with your hostname:

bbb-conf --setip yourhostname.tld

That's it! You have a working BigBlueButton server to do any form of conferencing. I assume that you installed BigBlueButton because you wanted to have online classrooms with moodle right? Proceed on to integrate Moodle and BigBlueButton.

Getting Moodle to work with BigBlueButton


You will need to download the moodle plugin for both applications to work together.

Install the moodle plugin under Site Administration > Plugins > Install plugins. Select the file you have download and upload it to the site with the "Install plugin from the ZIP file" button. Follow the instructions and you will eventually be at the configuration page for the BigBlueButton setup.

You need to enter the URL which to connect to BigBlueButton. For the shared secret, you will need to go to your BigBlueButton and type the following command:

bbb-conf --secret


Then copy and paste the shared secret to the moodle configuration. You may leave the rest of the configuration by its default, or configure as you desire.

Congratulations. You are now able to run Moodle classrooms with BigBlueButton.

About Me

Greetings Earthlings , Shurn the Awesomer is here to give you an awesome time.

This little site is a record of my life, opinions, and views. I'm mainly writing about Technology & Gadgets, Busting Creationist Myths, and other philosophical stuff.

This site is done using CakePHP.

Uptime

With this uptime, how much more can I be proud of to showcase to the world? This uptime monitoring is brought to you by StatusCake since 13th May 2017.

Copyright

I will always check for copyright usage before using any materials on my site. Whenever due, credit shall be given.

However, if you notice that I may have infringed on any copyright material. Please do not hesitate to contact me. All works of every artist deserves to be honoured and respected.