Setting up Active Directory with Samba on Ubuntu 16.04

Written on Tue, 21 June 2016

The last time I set up Active Directory as primary controller service was on 14.04. Since then, Ubuntu has been updated to 16.04.

The installation this time round is much easier than I expected. So here goes!

Prerequisites:

• You have Ubuntu 16.04 installed
• You have knowledge of linux commands
• You have some knowledge of how Samba works

We won't be using this server as file server. It's not recommended to put AD and file service in one server. That way, AD can continue to serve while file server is busy. It's not difficult to set up file server along with AD, but that won't be covered today. I'm just going to focus on setting up AD.

Most of the commands are done on root account. If you're not using root, be sure to add sudo before every command.

Setup Box Information

AD DC Hostname: ADSHURN
AD DNS Domain Name: shurn.me
Kerberos Realm: shurn.me
NT4 Domain Name/NetBIOS Name: shurn
IP Address: 10.0.0.2
Server Role: Domain Controller (DC)
Forwarder DNS Server: 10.0.0.1
File System: BTRFS

Your setup will be very different from mine, so you should adapt this tutorial to your own setup. I'll highlight the setup configuration with italics.

Step 1: Making sure Ubuntu is updated and current

It's always best to setup your AD on a well updated Ubuntu. Many bugs are fixed by the time you installed your Ubuntu.

apt-get update
apt-get dist-upgrade

Step 2: Download the required files

apt-get install attr build-essential libacl1-dev libattr1-dev libblkid-dev libgnutls-dev libreadline-dev python-dev libpam0g-dev python-dnspython gdb pkg-config libpopt-dev libldap2-dev dnsutils libbsd-dev attr krb5-user docbook-xsl libcups2-dev acl ntp ntpdate winbind

During Installation, your kerberos will ask you for information.

Realm=shurn.me
Server=shurn.me
Kerberos Login=shurn.me

Step 3: Setting up Static IP address

Active Directory requires static IP address.

nano /etc/network/interfaces

iface eth0 inet static
address 10.0.0.2
netmask 255.0.0.0
gateway 10.0.0.1
dns-nameservers 10.0.0.2 10.0.0.1
dns-search shurn.me

Later we will visit back this file to change the DNS information

Step 4: Setting up hostname

nano /etc/hostname

Put in the hostname of your AD

adshurn

Step 5: Reboot

Let's reboot the computer for all the configurations to take effect.

reboot

Step 6: Setting Host File

We need to make sure adshurn properly resolves to localhost at all times.

nano /etc/hosts

127.0.1.1 adshurn.shurn.me adshurn

Step 7: Setting up NTP

Network Time Protocol is the system that manages what time it is on your system, and it is important that our time is accurate for the proper functioning of Kerberos.

service ntp stop
ntpdate -B 0.ubuntu.pool.ntp.org
service ntp start

Step 8: Setting up Samba (finally)

So here's the main course of the entire tutorial. Let's install Samba.

apt-get install samba smbclient

We need to move the existing smb.conf file away so that we can properly generate a new file.

mv /etc/samba/smb.conf /etc/samba/smb.conf.orig

I usually use the interactive tool to set up Samba. You should too anyway.

samba-tool domain provision --use-rfc2307 --interactive

Realm: SHURN.ME
DOMAIN:''SHURN"
Server Role:dc
DNS backend:SAMBA_INTERNAL
DNS forwarder IP address:10.0.0.1
Administrator password: V3ryS3cur3P@$$w0rd

Step 9: Removing upstream DNS

Now, we remove the upstream DNS that we placed in the network configuration. resolv.conf is generated every time the system boots up so we need to ensure the next reboot, it searches the internal DNS. Samba is now managing all the DNS request.

nano /etc/network/interfaces

dns-nameservers 10.0.0.2

reboot

Step 10: Testing DNS works

Let's check that the DNS is working as expected. Remember to replace the setup information with yours.

host -t SRV _ldap._tcp.shurn.me
_ldap._tcp.shurn.mehas SRV record 0 100 389 adshurn.shurn.me

host -t SRV _kerberos._udp.shurn.me
_kerberos._udp.shurn.me has SRV record 0 100 88 adshurn.shurn.me

host -t A adshurn.shurn.me
adshurn.shurn.me has address 10.0.0.2

Step 11: Setting up Kerberos

As usual, let's move the original kerberos configuration to a safe place. Then we use the kerberos configuration provided by Samba.

mv /etc/krb5.conf /etc/krb5.conf.orig
ln -sf /var/lib/samba/private/krb5.conf /etc/krb5.conf

Step 12: Testing Kerberos and Samba

Let's make sure kerberos is working as expected

kinit administrator@SHURN.ME

Enter the password you used during Samba setup

klist

You should see valid krbtgt ticket.

smbclient -L adshurn.shurn.me -U 'administrator'

You should see netlogon and sysvol listed.

smbclient //localhost/netlogon -U 'administrator'

You should be able to authenticate properly.

Congratulations!

You should now have a properly set up Active Directory working in your environment. Pretty easy and straight forward isn't it?

Why port forwarding is bad?

Written on Mon, 30 May 2016

How many of our home routers have port forwarding for services like baby monitors and cameras? Maybe you set up a FTP server so that you can access your movies on-the-go.

But did you know, that is opening up your home systems for attack. It only takes 1 device to be compromised for the rest of the dominos to fall.

Why is it bad?

Devices such as baby monitors are built for the primary purpose of serving its function. It was never built as a network security device. It certainly has the basic security such as authentication, but that's usually the only thing.

On top of that, firmwares for these devices are not regularly patched, much less updated on the device itself. Any vulnerability discovered would likely stay vulnerable for a very very long time.

How do other devices get compromised?

It takes only 1 device to be compromised for the rest of the devices in the network to be compromised. Take for example, if you have a file server contain your accounting data. You have no port forwarding set for the server, thus, you assume that an incoming connection from the internet is not possible. But your port-forwarded IPCamera got compromised and it's on the same network as your file server. Since the IPCamera can connect to your fileserver, it's only a matter of time that the authentication on your file server is broken.

What should be done instead?

Instead of port forwarding directly to your devices, implement a hardware firewall in your network with VPN services. Firewalls are dedicated network security devices. They live for that one purpose and only that purpose. The software and/or the firmware of these hardware firewalls are regularly patched from vulnerabilities. Within its own settings, you are usually able to set automatic updates from the vendor. In your router, you might still need to port-forward to the firewall, which is fine if that is the only device you are port-forwarding. Hackers are stopped right at your firewall. Check your vendor if port-forwarding is required.

So how do you connect to your devices without port forwarding to it directly?

Here's where Virtual Private Network(VPN) comes into play. Most of us consumers who have used VPN before would think of it as a way to bypass country-restricted content like Netflix. But that's not the only thing VPN does. It connects your device on the internet back to your firewall, and allow it to connect to other devices as if it is on the same network.

A properly implemented VPN is extremely secure. It usually implements 2048bit RSA encryption, something that will take the current supercomputer more than 1 000 years to crack. Even if your encryption key is compromised, such as theft, you can immediately disable the encryption key to that device. The thief will no longer be able to access the VPN from that device.

So if you have baby monitors, IPcameras, or other hosting services for your personal use, use VPN.

Where do I get a hardware firewall?

There are commercial providers such as Fortinet that comes with annual subscription. It can be a tad-bit too expensive for small startups and home users, and maybe too complex to administrate for a network with less than 20 devices.

Other alternatives like Untangle offers a load more flexibility. Untangle is entirely open-sourced, so techies will love it. For the everyday consumer, you will love how easy it is to use.

The open sourced edition is free, as in beer, forever. It uses a freemium model where only enterprised services have an annual fee. The only cost you have to worry about, is the hardware. You can reuse most old hardware as Untangle firewall. You probably just need to install another network card so that you have 2 ethernet ports. You can work with 1 etherport, but it's complex to setup and less secure. Check out the minimum hardware before you build it. If that's not an option, you can buy an affordable appliance that comes with 1 year warranty. Annual subscription and extended warranty is optional.

Untangle uses OpenVPN for VPN services. It's an already proven technology that is able to bypass certain port blocks, as it uses the standard port 80 instead of specialised port that can be easily blocked by ISPs and wifi providers. Just create a client and you will be able to download the encryption certificate for your devices. If you are using Windows, it comes with the client itself. Otherwise, you might want to read the documentation for your device, such as android or iOS, on how to instead the certificates. You might need to port forward TCP/UDP 1194 on your router to your firewall, and that should ever be the only port forwarded. Check with your ISP, vendor, or network administrator if this is required.

Once you are connected back to your LAN, freely browse to your device IP address.

Why is this so important?

We are living in a world getting more connected than before. Our devices at home are more internet-enabled than before. We are already at the age of Internet of Things, where every little thing, from our kids' toys to the common refrigerator, is now connected to the internet.

The sole responsibility of network security does not lie with the manufacturers of these things. We are equally responsible for the network security of our homes. Take this simple step to secure your home or small office network.

Let's Encrypt the internet

Written on Fri, 20 May 2016

Every website should have https option. In fact, it should be the default. But how many times have we seen that https cert costing a lot, and even the free certs require 30 days regular renewal. Some offer 90 days renewal, but that's aside the point. Certs require maintenance.

But we can thank Internet Security Research Group, a public benefit corporation, for providing the internet with free and hassle-free certificates. ISRG is sponsored by many organisations, from non-profits to Fortune 100 companies. A full list of sponsors can be found here.

Important

One important thing to note is that this is not Extended Validation(EV) certificates. EV certificates can be obtained from Comodo. These certificates require additional verification beyond the technology, which is why it's particularly important if you want to conduct online transactions. Your customers need to know that the website(YOU!) they are transacting with, is trusted.

Alright, so let's get started to install certs for our web servers. If you had followed my tutorial on setting up a web hosting site, this tutorial is exactly for you. If you had set up your own web server under different configuration, the tutorial here should still apply to you for most part. For more information, you can go here to find a specific instruction for your build.

By the end of this tutorial, you will have:

• certbot installed
• All sites in your server instance to have https enabled, if you didn't proceed to revoke.
• All Certificates installed will automatically renew every 90 days.
• Bonus: If you have piwik installed from previous tutorial, you can force SSL on it.

What you need:

• You need to have a web hosting site already running. You can follow my tutorial on setting up a web hosting site to get you started.
• Some linux knowledge

Installation

Let's start with installing the certbot to Ubuntu 16.04. SSH into your server. All commands are done on root account. Prepend sudo if you are not in root account.

apt-get install python-letsencrypt-apache

Automated installation

Let's run the installation programme:

letsencrypt --apache

On the first screen, it will prompt you for which sites you would like to install certs to. Choose all that you want to. As for me, I picked all my sites.

Next, you have to enter your email address.

Be sure to read all the legal terms and conditions of usage before you proceed to install the certificates. Especially the rate limits. If you agree, just select agree on your SSH console.

Now it's going to prompt you whether to set https as the only connection or allow both http and https. For me, I prefer to use both and instead leave it to individual application to redirect to https. Choose the option which you prefer.

That's it for the installation. You are set to use https. You can verify your ssl is working with https://www.ssllabs.com. Your Certificate will also last you 90 days. It's also a good idea to regularly backup /etc/letsencrypt folder.

In case you can't connect to https, be sure to check your aws security groups and allow port 443 to your instance.

Automated Certificate Renewal

Since your certificate will only last you 90 days, you need to regularly renew your cert. Now, here's the part where Let's Encrypt does magic better than most other free certificate providers. It will automatically renew for you, which is as good as a permanently free service.

Let's do a test run on certificate renewal to see if any errors occur:

letsencrypt renew --dry-run --email [YOUR-EMAIL] --agree-tos

Make sure you clear away any errors if you encounter any.

If there is no error, you can proceed to set automatic renewal through cronjob:

crontab -e

Add the following to the cronjob entries:

30 */12 * * * letsencrypt renew --email [YOUR-EMAIL] --agree-tos

This will run the renewal twice every day, recommended by their documentation. It is advisable that you run the renewal process on a random minute. I guess this is to ease their server load for being hit by renewal request when the clock strikes a particular minute.

That's it! You now have a permanently free SSL cert for all your websites. If this service has benefited you, do consider a small donation.

Revoking a certificate

If you ever need to revoke a certificate, here's how you can do it:

letsencrypt revoke --cert-path /etc/letsencrypt/[path-to-your-cert]

If your cert is placed elsewhere, do modify the path accordingly.

Your cert will no longer authenticate with the Certificate Authority after awhile. That's it.

Conclusion

This is by far the easiest way to install certificates I've ever experienced. It practically does everything for me. I don't even need to know how to generate keys, or configure anything on Apache. It just works.

Great job, Let's Encrypt

Bonus: Force SSL for Piwik

Did you follow my tutorial on Piwik Installation? If you did, you can force Piwik to use https everytime. It's highly recommended that you do so, so that you can protect your login credentials from being in the clear during transaction.

Browse to your piwik installation configuration folder

cd [path-to-piwik-installation]/config

Edit the config.ini.php

nano config.ini.php

Add the following under [General]: (or edit the line if the settings already exists)

[General]
force_ssl = 1

That's it! Your Piwik installation will now use https whenever it is browsed. You have made your site secure and the internet a little bit safer.

Piwik: Private Analytics

Written on Fri, 13 May 2016

Piwik is easily the best analytics for organisations who have a privacy policy that doesn't allow them to share information with third party. If you've tried Google Analytics, who will find that Piwik offers very similar functionalities. The biggest downside is probably that Google actually integrates its services seamlessly, while piwik is a purely standalone product. But the biggest advantage is that Piwik is entirely open sourced, so you can go ahead to modify or extend its functionalities.

Requirements

Minimum:

• Webserver such as Apache, Nginx, IIS, etc
• PHP5.3 or greater
• MySQL version 4.1 or greater, or MariaDB
• PHP extension pdo and pdo_mysql, or the mysqli extension

Recommended:

• Webserver such as Apache, Nginx, IIS, etc
• PHP7
• Latest MySQL or MariaDB
• PHP extension pdo and pdo_mysql, or the mysqli extension

Tip: Check out my other blog post for more information on setting up a web hosting with PHP7 and infinitely growing hard disk space.

Preparing the Server

Now I assume you have followed my previous blog post and installed PHP7 on Ubuntu 16.04. If not, do check it out! Let's prepare the server for Piwik installation.

Starting with the required libraries:

apt-get install php7.0-curl php7.0-gd php7.0-cli php7.0-dev libgeoip-dev

Interestingly, at the time of writing, GeoIP extension is not supported in PHP7. So we are going to need some help. Fortunately, Zakay from GitHub has compiled a solution. Let's hope there will be an official release of GeoIP extension for PHP7 soon.

We are going to get the codes from github:

git clone https://github.com/Zakay/geoip.git
cd geoip

Next, we compile it:

phpize
./configure --with-php-config=/usr/bin/php-config
make
make install

Now it is ready to be used. Let's put the settings in php.ini so that it will during runtime.

touch /etc/php/7.0/mods-available/geoip.ini
echo "extension = geoip.so"> /etc/php/7.0/mods-available/geoip.ini
phpenmod geoip
service apache2 restart

I hope you still have your phpinfo.php file in your webroot. Run the script in your browser and you should see GeoIP installed.

Next we configure your mysql so that it has a username for itself. It's always a good security measure that you assign each app to a single user and not shared.

mysql -p

Then enter your root password. If you have another username, do use the following command instead:

mysql -u anotherusername -p

You should now be at the mysql console. Enter the following:

CREATE DATABASE piwik_db_name_here;
CREATE USER 'piwik'@'localhost' IDENTIFIED BY 'password';
GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES ON piwik_db_name_here.* TO 'piwik'@'localhost';
exit

Do substitute the names accordingly. Otherwise, your future administrators might think you're lazy.

Once you are done with that, you are absolutely good to proceed to piwik installation.

Installing Piwik

Now you are ready to download the piwik installation. You can download it to your computer, unzip, and upload it via ftp or sftp. But for pure linux sake, here's the commands to do it:

apt-get install unzip
wget http://builds.piwik.org/piwik.zip
unzip piwik.zip
mv piwik/* /var/www/html
rmdir piwik
rm How\ to\ install\ Piwik.html piwik.zip

The last few commands to remove is for cleaning up no longer needed files. It's always a good habit to clean up after yourself. Keep servers neat and tidy is important. The piwik is installed in the default webroot. If you are using it as a virtual host, you must configure apache accordingly, which is not covered in this tutorial.

If you browse to your website in your web browser, you should see your piwik installation. If the page shows an error /tmp/cache/tracker, you need to ensure that the tmp folder has enough write permission. It's also recommended that you keep the entire installation writable by apache so that piwik can do self-upgrade.

Go ahead to install piwik by following the instructions. And you're done! Congratulations!

Setting up a web hosting site with infinite storage

Written on Thu, 5 May 2016

Running a web hosting site is already quite a challenge for most beginners. What if you're task with running a site that could potentially be hosting a ton of image and videos? Well yeah, you could use S3 storage from Amazon Web Services. But that application just doesn't support S3 storage and modifying the application ain't an option.

So the big question, how do you have a server storage that can continually grow without downtime involved?
Welcome to the world of LVM, Logical Volume Management.

In this tutorial, I'll be showing you how you can setup a Ubuntu 16.04 on AWS with infinite growing storage space, by regularly adding EBS volumes to your instance whenever you need.

---

By the end of this tutorial, you will have:

• Ubuntu 16.04
• 3 EBS volumes attached with 1TiB space available for web hosting
  • 8GiB for OS and softwares
  • 2x 500GiB volume for web hosting, formatted with BTRFS
• 1 Volume Group
• 1 Logical Volume
• LAMP Stack Installed

What you need:

• Amazon Web Service account
  • Some knowledge administrating the AWS console
• Basic linux administration experience

What will not be covered:

• Database tables for infinite growth. Use RDS for that. But if you absolutely need it, the steps are quite similar to setting up apache logs section.

---

Setting up the instance

Step 1: Choose AMI
Locate the right AMI for your region with the AMI Locator. I'm using ami-840910ee because I want my instance to be located in North Virginia. Try to use 16.04 LTS since this is what the tutorial is all about. For older versions, additional steps are required as you progress through this tutorial, which will not be covered.

Step 2: Choose Instance Type
Instance type: t2.nano (Choose the type that suits your production environment)

Step 3: Configure Instance
Configure accordingly. If you're not sure, usually the default settings will do fine. Read more at AWS documentation.

Step 4: Add Storage
Here's the important part of the tutorial. The AMI you selected uses 8GiB to run. In most cases, this is more than enough to run the OS and the application you want to use. In our case, LAMP stack won't even take up 1GiB. By the end of the tutorial, you will still have about ~6GiB to play around with.

Next, let's add another volume to the instance and give it 500GiB, more if your production environment needs it. This will be the volume for web hosting. For volume type, choose the one that suits your production environment. For tutorial purposes, "Cold HDD (SC1)" is good enough.
Volume Type: Cold HDD (SC1)
Size: 500GiB

Step 5: Tag Instance
Configure the tags you need. It's ok to leave blank if you're not sure.

Step 6: Configure Security Group
Configure the security group. You will definitely need at port 22, 80, and 443. Adjust and add settings as your application needs.

Step 7: Review Instance Launch
Have a final check on the settings and configuration. If everything is good, go ahead and launch the instance. It will take about 5 minutes. So meantime, grab a cup of tea.

---

Configuring the web host

SSH into your newly launched instance. Your username is "ubuntu". You will need your keyfile that was generated for you to login.

It's a good time to run all the necessary procedures to keep your Ubuntu up to date with the latest security patches. All codes are running with root access for convenience. If you prefer not to run on root, be sure to prepend "sudo" in front of most command.

sudo su
apt-get update
apt-get -y dist-upgrade
apt-get -y autoremove

After an update, it's usually a good idea to give your instance a reboot

reboot now

SSH back into your instance. Let's install the LAMP stack.

sudo su
apt-get -y install lamp-server^

You will be promted to give your mysql root user a password. Give it a password even if you don't intend to use it, as a good practice.

By this point, you can browse to the instance IP address with your web browser. You will see a Apache2 Ubuntu Default Page, with the world's most famous cliche: It works!

If you don't see this, you might want to restart this tutorial or trace back where you might have gone wrong.

---

Configuring your infinite storage

Finally, the main point of this tutorial. You will need to start off with looking at your attached hard disk.

parted -l

It should look something like these if you've followed the previous steps accordingly.

So our hard disk is /dev/xvdb. We are going to let LVM manage this disk.

pvcreate /dev/xvdb

Next, we are going to create a volume group called "inifinitestoreVG" with the hard disk.

vgcreate infinitestoreVG /dev/xvdb

Finally, we will create a logical volume out of the volume group called "infinitestoreLV" with 499.9GiB. We can't use all 500GiB because LVM just seems to need some space for itself. 0.1GiB ain't significant when you can infinitely add storage anyway.

lvcreate -n infinitestoreLV -L 499.9g infinitestoreVG

Let's look at your Logical Volumes.

lvdisplay

Your new logical volume is /dev/infinitestoreVG/infinitestoreLV.

We will need to format this new volume with a filesystem. We are going to use BTRFS because of its cool features. A word of caution: if you are using older versions of linux, BTRFS might still be experimental. Use ext4 if you are using older ubuntu versions.

mkfs.btrfs /dev/infinitestoreVG/infinitestoreLV

Let's mount the new volume in at location where apache host its websites.

mount /dev/infinitestoreVG/infinitestoreLV /var/www

Right now, it's an empty hard disk. So let's create back the site to show that we can use the new hard disk for hosting.

mkdir /var/www/html
nano /var/www/html/index.php

We are going to make a PHP page so that we can output something.

<?php phpinfo();

By this point, if you have done everything right, you can see a PHP info output on your browser when you browse to your instance's ip address.

Congratulations, you have successfully run a web hosting site on an ever growing volume group whenever you want it to.

But that's not the end yet. What happens when you reboot? You will have to mount the volume group again manually. To automate this process, let's modify the fstab:

nano /etc/fstab

Add the following line:

/dev/infinitestoreVG/infinitestoreLV /var/www/ btrfs defaults 0 0

Now if you reboot and browse back to the IP address, you will still see the php output. If you see the default ubuntu page instead, the auto-mount is not working.

reboot now

---

Optional: Apache Logs

Generally speaking, you would do fine if you stop right here. But if you want to take one step further, you can also ensure the all apache logs are also located in the Logical Volume. This is very useful if you intend to keep your logs for eternity!
First, create a folder in the Logical Volume:

mkdir /var/www/log

Now, you need to modify the settings of apache to generate logs in the new folder

nano /etc/apache2/envvars

Look for the following line:

export APACHE_LOG_DIR=/var/log/apache2$SUFFIX

Change it to:

export APACHE_LOG_DIR=/var/www/log$SUFFIX

Now, restart apache:

service apache2 restart

---

Expanding your storage

Has it come to the time to increase your space for web hosting? You have nothing to fear, if you've followed the previous steps correctly.
Let's start with giving yourself a new 500GiB, or more, volume in your AWS console. It is recommended that you keep the volume type consistent will the ones you currently have. I was using "Cold HDD (SC1)" before, so I will use back the same one. You will need to create it in the same zone as your instance.

Next, we need to attach it to the instance.
Device: /dev/sdc

Now go back to the SSH. Let's have a look at the hard disk detected by the OS.

parted -l

You should see "/dev/xvdc".

Let's ensure the device is known to LVM.

pvcreate /dev/xvdc

Let's give the device to infinitestoreVG.

vgextend infinitestoreVG /dev/xvdc

Extend the volume all the way to 1TiB.

lvextend /dev/infinitestoreVG/infinitestoreLV /dev/xvdc

Last step before we can finally use the new space we added. The file system must recognise the newly added space.

btrfs filesystem resize max /var/www
df -h

Now, you should have 1000G worth of space in /var/www, or 1TiB.

---

Congratulations

You now have a web hosting server where you can grow your hosting space infinitely. You can attached unlimited amount of EBS volumes to an instance. Just take note that there is a limit of EBS volumes you are allowed to create. And also, BTRFS has a limit of 16EiB, which is 16 384PiB, which is 16 777 216 TiB. You have a limit of 9 223 372 036 854 775 807 files allowed. You're probably not going to reach that limit anyway, so that's not something you need to worry so soon.

You can read up more information about:

• Logical Volume Management
• BTRFS: You should really take advantage of the features available in BTRFS for your hosting. Particularly, the quota management feature available.