Deep MNIST learning

Written on Thu, 25 August 2016

I have resumed my Machine Learning learning. Yea, bad pun! After writing a long blog post previously on the Origins Debate and many other things I need to do, I'm starting to write machine learning codes. Honestly, I'm still absolutely clueless about the concept of Machine learning and its terminology.

What's shape?
What's convolution?
What's weight?

Questions and more questions.
I'm following the tutorial at tensorflow tutorials for Experts. Ironic, I'm no expert.

Here's the codes I've written so far.

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

import tensorflow as tf
sess = tf.InteractiveSession()

x = tf.placeholder(tf.float32, shape=[None, 784])
y_ = tf.placeholder(tf.float32, shape=[None, 10])

W = tf.Variable(tf.zeros([784,10]))
b = tf.Variable(tf.zeros([10]))

sess.run(tf.initialize_all_variables())

y = tf.nn.softmax(tf.matmul(x,W) + b)

cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))

train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

for i in range(1000):
batch = mnist.train.next_batch(50)
train_step.run(feed_dict={x: batch[0], y_: batch[1]})

correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(accuracy.eval(feed_dict={x: mnist.test.images, y_:mnist.test.labels}))

def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)

def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)

def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')

def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

W_convl = weight_variable([5, 5, 1, 32])
b_convl = bias_variable([32])

x_image = tf.reshape(x, [-1,28,28,1])

h_convl = tf.nn.relu(conv2d(x_image, W_convl) + b_convl)
h_pooll = max_pool_2x2(h_convl)

W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])

h_conv2 = tf.nn.relu(conv2d(h_pooll, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

W_fcl = weight_variable([7 * 7 * 64, 1024])
b_fcl = bias_variable([1024])

h_pool2_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64])
h_fcl = tf.nn.relu(tf.matmul(h_pool2_flat, W_fcl) + b_fcl)

keep_prob = tf.placeholder(tf.float32)
h_fcl_drop = tf.nn.dropout(h_fcl, keep_prob)

W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])

y_conv = tf.nn.softmax(tf.matmul(h_fcl_drop, W_fc2) + b_fc2)

cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
sess.run(tf.initialize_all_variables())
for i in range(20000):
batch = mnist.train.next_batch(50)
if i%100 == 0:
train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_: batch[1], keep_prob: 1.0})
print("step %d, training accuracy %g"%(i, train_accuracy))
train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

print("test accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))

I don't even know if I'm doing it correctly.

Simplifying Active Directory password on Samba

Written on Wed, 27 July 2016

I often find the password security for simple usage a little overkill. Of course, the security is deliberately set high for our protection, and is highly discouraged to be lowered. I'm not endorsing lower password security. But if you need to simplify your Active Directory password for your user accounts, use these commands:

sudo samba-tool domain passwordsettings set --complexity=off
sudo samba-tool domain passwordsettings set --history-length=0
sudo samba-tool domain passwordsettings set --min-pwd-age=0
sudo samba-tool domain passwordsettings set --max-pwd-age=0

By this, password complexity won't be required, you can reuse old passwords, and you never have to change password due to expiry. Much less administration, if you ask me.

Blog Post has navigation

Written on Sat, 2 July 2016

As usual, nothing to be impressed about. I've placed navigation on my blog post on the right column and at the end of the post. In my previous post, I mentioned that I used a technique to implement the neighbour function found in CakePHP2 to CakePHP3. So here's the result.

So now what's next? SEO Images. If I'm going to announce my blog, I'm going to have to make sure it can be properly indexed by social media and search engines.

After which, I really hope I can monetise a little bit from my own blog. Just to sustain the hosting cost. I have a placeholder on the right column that says related content just for advertising, and maybe some of my related own post after I have written enough content on my blog...

CakePHP3 equivalent of neighbor function in CakePHP2

Written on Thu, 30 June 2016

Strangely, CakePHP3 doesn't have the neighbour function that CakePHP2 used to have. I thought it was such a useful function. Now that I want to use it, CakePHP3 that I'm using doesn't support it anymore.

I'm trying to use it to get the previous and the next blogpost for my navigation quest. After looking around for solution, I'm inspired by this blog post. I've modified the codes to use entity instead of using it as a model, as it's more fitting to be.

public function neighbours() {
$blogposts = TableRegistry::get("MyBlogPostTable");

$prev = $blogposts->find("all", [
"conditions" => [
"id" => $this->id - 1,
],
])
->first()
;

$next = $blogposts->find("all", [
"conditions" => [
"id" => $this->id + 1,
],
])
->first()
;

return [
"prev" => $prev,
"next" => $next,
];
}

Now, don't criticise my codes yet. Firstly, I assume that ID is always retrieved. Secondly, my ID is already natural number. Thirdly, my ID is auto-incremental and in sequence with time.

If you're going to use a code similar to mine, do adjust it to your use case.

Tutorial: Setting up Tensorflow in Ubuntu 16.04

Written on Mon, 27 June 2016

So these days I'm just learning up on Machine Learning. Quite a fascinating subject. But I'm no data scientist (yet!) so it's quite daunting what Machine Learning is capable of.

Today's tutorial will just be about setting up TensorFlow on your Ubuntu machine. I won't go in depth about machine learning algorithm or help you build a Terminator T-5000, for obvious reasons.

In my machine build, I do not have a Graphic Card, so I won't be touching on graphic card related configuration. But I sure don't mind you sponsoring me a Quadro M6000. I'm also using 64bit computer. Seriously, why would you have a 32bit computer to run Machine Learning?

Step 1: Installing Python3

There's no reason to use a distribution no longer receiving upgrades. Python2 will still be maintained but why not use the one built for the future? Let's install Python3.

apt-get install python3-pip python3-dev

Step 2: Installing Tensorflow

export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.9.0-cp35-cp35m-linux_x86_64.whl
pip3 install --upgrade $TF_BINARY_URL

This will take some time. Go make yourself some coffee!

Step3: Testing your installation

Let's execute python to make sure your installation works.

python3
>>>import tensorflow as tf
>>>hello = tf.constant('Hello, TensorFlow!')
>>>sess = tf.Session()
>>>print(sess.run(hello))
Hello, TensorFlow!
>>>a = tf.constant(10)
>>>b = tf.constant(32)
>>>print(sess.run(a + b))
42
>>>exit()

Step 4: Run your first Tensorflow Model

Tensor is nothing if you don't run any machine learning. Let's run your first Tensorflow Model.

cd /usr/local/lib/python3.5/dist-packages/tensorflow
python3 -m tensorflow.models.image.mnist.convolutional

Congratulations! You have run your first Machine Learning model. This model will take quite awhile to complete. Took me more than 40 minutes on my slow machine without GPU. If you have a GPU installed, I'm sure it will be a whole load faster.