Current OS = Linux Mint 13

INTRO

The aim of this blog. You know how something challenges you and you google away, find a fix with some 'trial and error' and then in the future someone asks about how you did it, or you need to alter/re-do it at a later date but you have forgotten what little trick you did to accomplish it ? Well my aim is to keep a track of what I am working on and methods I have used here. And now, I can access it easily, it can be google indexed for others and I will have a URL to send others for problems I cant recall off hand how I fixed them. I hope you find this site useful.

21-07-2014 12:13

Networking tips and tricks

This is a good article for basics using network manager. A good read for anyone using Network Manager. The one issue is the hosts file editing. It is better to put the FQDN first after the IP address and then any short names after that as the first name after the IP is what the system uses to resolve it's own hostname. So the command hostname -f won't work if you have your systems FQDN second or later on it's 127.0.0.1 line. It is just a good habit to be in.


Posted by DaveQB | Permanent Link | Categories: IT

23-06-2014 15:57

AWS Sydney Region network throughut

I did some iperf testing on the two Availability Zones (AZ) in the Sydney AWS region. I have 4 gluster servers a m1.medium and m3.large in ap-southeast-2a and a m1.medium and m3.large in ap-southeast-2b. So I did a cross AZ iperf test between like servers.

m1.medium 2a > m1.medium 2b

[ ID] Interval Transfer Bandwidth 
[ 4] 0.0-10.0 sec 220 MBytes 184 Mbits/sec 
[ 7] 0.0-10.0 sec 188 MBytes 157 Mbits/sec 
[ 6] 0.0-10.0 sec 193 MBytes 161 Mbits/sec 
[ 8] 0.0-10.1 sec 262 MBytes 218 Mbits/sec 
[SUM] 0.0-10.1 sec 863 MBytes 719 Mbits/sec 

[ 4] 0.0-10.1 sec 219 MBytes 181 Mbits/sec 
[ 6] 0.0-10.2 sec 227 MBytes 187 Mbits/sec 
[ 7] 0.0-10.2 sec 197 MBytes 162 Mbits/sec 
[ 3] 0.0-10.2 sec 221 MBytes 182 Mbits/sec 
[SUM] 0.0-10.2 sec 864 MBytes 712 Mbits/sec 


m3.large 2a > m3.large 2b

[ ID] Interval Transfer Bandwidth 
[ 7] 0.0-10.0 sec 222 MBytes 185 Mbits/sec 
[ 3] 0.0-10.0 sec 66.0 MBytes 55.1 Mbits/sec 
[ 6] 0.0-10.1 sec 96.6 MBytes 80.6 Mbits/sec 
[ 8] 0.0-10.1 sec 56.8 MBytes 47.3 Mbits/sec 
[SUM] 0.0-10.1 sec 441 MBytes 368 Mbits/sec 

[ 7] 0.0-10.1 sec 140 MBytes 116 Mbits/sec 
[ 3] 0.0-10.1 sec 162 MBytes 134 Mbits/sec 
[ 6] 0.0-10.2 sec 79.1 MBytes 65.3 Mbits/sec 
[ 5] 0.0-10.2 sec 57.4 MBytes 47.4 Mbits/sec 
[SUM] 0.0-10.2 sec 438 MBytes 362 Mbits/sec

Not that good, not gigabit speeds, but not terrible.

Now keeping the test inside the same AZ.

m1.medium 2b > m3.large 2b

[ ID] Interval Transfer Bandwidth 
[ 7] 0.0-10.0 sec 105 MBytes 87.6 Mbits/sec 
[ 6] 0.0-10.0 sec 161 MBytes 135 Mbits/sec 
[ 5] 0.0-10.1 sec 93.9 MBytes 78.3 Mbits/sec 
[ 8] 0.0-10.1 sec 86.4 MBytes 71.9 Mbits/sec 
[SUM] 0.0-10.1 sec 446 MBytes 371 Mbits/sec 

[ 5] 0.0-10.2 sec 143 MBytes 118 Mbits/sec 
[ 4] 0.0-10.2 sec 93.5 MBytes 77.0 Mbits/sec 
[ 7] 0.0-10.2 sec 119 MBytes 98.2 Mbits/sec 
[ 6] 0.0-10.2 sec 89.0 MBytes 73.3 Mbits/sec 
[SUM] 0.0-10.2 sec 445 MBytes 366 Mbits/sec 

Not much better.


Posted by DaveQB | Permanent Link | Categories: IT

21-05-2014 13:32

How to generate a HmacSHA512 from the shell

This is ripped right from here but I am copying here for my own (and others) reference.

I realise this isn't exactly what you're asking for, but there's no point in reinventing the wheel and writing a bash version. You can simply use the openssl command to generate the hash within your script.

[me@home] echo -n "value" | openssl dgst -sha1 -hmac "key"
57443a4c052350a44638835d64fd66822f813319
Or simply:
[me@home] echo -n "value" | openssl sha1 -hmac "key"
57443a4c052350a44638835d64fd66822f813319
Remember to use -n with echo or else a line break character is appended to the string and that changes your data and the hash. That command comes from the OpenSSL package which should already be installed (or easily installed) in your choice of Linux/Unix, Cygwin and the likes. Do note that older versions of openssl (such as that shipped with RHEL4) may not provide the -hmac option. As an alternative solution, but mainly to prove that the results are the same, we can also call PHP's hmac_sha1() from the command line:
[me@home]$ echo '' | php
57443a4c052350a44638835d64fd66822f813319
Edit: One could use printf rather than echo -n.


Posted by DaveQB | Permanent Link | Categories: IT

12-03-2014 17:51

AWS-CLI

I have been running all the different cli API tools for EC2 access at work. It is handy and needed for scripting purposes, but so slow, even the command line completion is slow. For example, doing an ec2-describe-instances was taking 1min and 14 seconds, with the new aws-cli tools it takes 1.87 seconds.. Well worth the hassle.


Posted by DaveQB | Permanent Link | Categories: IT

07-03-2014 11:25

Denyhosts ignores some whitelisting

I had issues at work with some IPs that fall into an IP wildcard range that was in my denyhots whitelist being blocked by denyhosts. It turns out they were Class B networks and denyhosts wildcard white listing only accepts Class C networks. Although written incorrectly here you undestand what it is saying. The examples do only show Class C networks, but it doesn't explicitly say it ONLY accepts Class C networks, hence why it was hard for me to troubleshoot this issue.


Posted by DaveQB | Permanent Link | Categories: IT

25-11-2013 18:34

Time calculations in shell

I have had needs where I needed to calculate the time something has taken to run. It can be hard when going over a date in the calendar. To add to that, some shells don't allow floating point calculation too. So I sat down and worked out a scrpt that when given a start second and a finish second (like seconds since epoch), it can show the difference in HOURS:MINUTES. Feel free to modify as you please, GPL3 of course.

#!/bin/bash

# Capture the epoch start and then finish time.
#SLEEPSTOP=1385183548
#SLEEPSTART=1385180248

# Or give it the absolute seconds.
SLEEPSTOP=0
SLEEPSTART=1440

SLEEP=$(($SLEEPSTOP-$SLEEPSTART))
HR=$(($SLEEP/60/60))
MINS=$(echo $(($SLEEP/60*100/60))|rev|cut -c-2|rev)
MINS=$((MINS*60/100))

echo ${HR}:${MINS}


Posted by DaveQB | Permanent Link | Categories: IT

25-10-2013 17:59

Allow root ssh from selected machines

Sometimes you need to allow root ssh access. But this is not a great idea; I disallow root ssh on all my systems, instead finding alternative (often much more difficult) solutions such as running two or more ssh daemons and their config files and control access to them with iptables. Convoluted.

Recently I found out how you can use the Match statement to selectively allow root ssh (and even limit it to PubkeyAuthentication only). It is rather simple and reading the sshd_config man page explains more. Below is a self explaining example. The without-password actually means PubkeyAuthentication only and password auth is disabled. It does look a bit scary if you didn't know that. I did find that only having the Address 127.0.0.1 didn't allow root ssh access to localhost.

Match Address 127.0.0.1,23.4.76.129,8.8.8.8
        PermitRootLogin without-password

Match Host localhost
        PermitRootLogin without-password

Posted by DaveQB | Permanent Link | Categories: IT

09-10-2013 15:14

FUDForum emails blocked by Hotmail and Gmail

I suddenly started having with issues where Gmail and Hotmail were blocking emails sent by my forum using my email server. Emails sent from this same email server by me using Thunderbird or anyone else, where not blocked. So somehow it was the way FUDforum was constructing the email headers. The reasons for blocking where RFC2822 (Gmail) and RFC5322 (Hotmsil). It turns out RFC2822 was superseded by RFC5322 (as RFC822 was superseded by RFC2822).

I found an online email header checker (here is one although not what I used http://mxtoolbox.com/public/tools/emailheaders.aspx). Comparing the headers Thunderbird would create verse FUDforum found that they both had errors in the checker but FUDforum headers had this: "WARNING: duplicate header 'From' at line 21". So I looked into the code for FUDforum and found where it was adding a From header and commented out. This file is $DATA_FOLDER/src/iemail.inc.t which will fix this for all of your themes. Recompiled the theme and voila! Fixed.

All of this is in this thread: http://fudforum.org/forum/index.php?t=msg&goto=183103

PATCH:


--- src/iemail.inc.t-orig    2013-09-23 04:14:00.680024261 +0000
+++ src/iemail.inc.t         2013-09-23 04:13:43.409824908 +0000
@@ -63,7 +63,8 @@
        if (strpos($header, 'MIME-Version') === false) {
                $extra_header = "\nMIME-Version: 1.0\nContent-Type: text/plain; charset=utf-8\nContent-Transfer-Encoding: 8bit". $header;
        }
-       $header = 'From: '. $from ."\nErrors-To: ". $from ."\nReturn-Path: ". $from ."\nX-Mailer: FUDforum v". $GLOBALS['FORUM_VERSION']. $extra_header. $header;
+       //$header = 'From: '. $from ."\nErrors-To: ". $from ."\nReturn-Path: ". $from ."\nX-Mailer: FUDforum v". $GLOBALS['FORUM_VERSION']. $extra_header. $header;
+       $header = $extra_header. $header;
 
        $body = str_replace("\r", '', $body);
        if ($munge_newlines) {

UPDATE: I found the email header checker site I used: http://www.apps.ietf.org/content/message-lint


Posted by DaveQB | Permanent Link | Categories: IT

12-08-2013 14:09

x11vnc is repainting the entire screen way too often

I don't often need to, but sometimes I need to get access to the GUI of a computer. Mostly my desktop. I do a port forward over ssh and and then launch x11vnc, attaching to it over the tunnel with a VNC viewer. I have a nice x11vncrc file in my home that I use.

When I switched from PCLinuxOS to Linux Mint about 12 months ago I had an issue with this work flow. The screen would do a full screen repaint every few seconds. The result was the screen was in a constant state of black and slowly repainting and then repainting again. This meant it was unusable. This guy here as the same issue..

I tried what solved it for him, adding the -nodpms switch, doubting it would work as it is a not a screensaver issue for me. Well it fixed it. I have no idea why. If anyone does, please comment.


Posted by DaveQB | Permanent Link | Categories: IT

05-08-2013 20:58

Setting up LVM on a luks encrypted partition.

I have let this blog go a bit, ok a lot. Maybe because I have been lazy and not documented my recent findings or perhaps I am getting so good I am know all the answers. Yes, the answer is obvious.

Today I setup a laptop work gave me to use. We use Ubuntu 12.04 on the servers here, so I figured it would be smart to use Ubuntu 12.04 (or derived distro). I love KDE and I am using Mint 13 on my desktop, so made sense to go with that. What I quickly found is that the installer is lacking any options for encrypting your disk.

Off I went to find a tutorial on this. Three main pages I used was: https://www.martineve.com/2012/11/02/luks-encrypting-multiple-partitions-on-debianubuntu-with-a-single-passphrase, http://blog.lifebloodnetworks.com/?p=1348 and https://help.ubuntu.com/community/EncryptedFilesystemLVMHowto. The last link is close to what what I wanted (page may be deleted, it is saying!) but I also wanted to go with GPT because...why not, it is 2013. So I will cover the steps here.

STEP 1

Launch a terminal and install the needed software before starting the Mint installer.
sudo -i
apt-get install lvm2 cryptsetup gdisk
gdisk /dev/sda
Setup a GPT partition table. I can't remember the command, it wasn't hard. Now setup 3 partitions. 1 will be ef02 type and only needs to be 1M. This is for grub-bios. It stores the stage 2 files to allow booting on systems that don't support EFI and do a BIOS boot. GPT doesn't have an alloated space for these files, hence the partition. Then partition 2 will be the boot partition, type 8300. I set it to 500MB. Then last will be the LVM pv, type 8e00, using all remaining space.

STEP 2

Next we setup the disk and partitions.
cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 /dev/sda3
cryptsetup luksOpen /dev/sda3/ system
And now the LVM on top of the encrypted partition (/dev/sda3).
pvcreate /dev/mapper/system
vgcreate luks /dev/mapper/system
lvcreate -n swap -L 8G
lvcreate -n slash -L 12G
lvcreate -n home -l 100%FREE
Then I found the installer wouldn't recognise these logical volumes unless they were formated so...
mkfs.ext2 /dev/sda2
mkfs.ext4 /dev/luks/slash
mkfs.ext4 /dev/luks/home
mkswap /dev/luks/swap

STEP 3

Now we proceed with the installer. When it comes time to do the partitioning, select Manual. Be sure match the right paritions with the right mount points In my example:
/dev/luks/slash (/)
/dev/luks/home (/home)
/dev/sda2 (/boot)
/dev/luks/swap (swap)
Don't reboot once the install has completed.

STEP 4

Now we setup the new install to be aware of and use the new disks.
sudo -i
mkdir a
mount /dev/luks/slash a
mount /dev/sda2 a/boot
mount -B /dev a/dev
mount -B /dev/pts a/dev/pts
mount -B /sys a/sys
mount -B /proc a/proc
chroot a
apt-get update
apt-get install lvm2 cryptsetup

STEP 5

You can check your new /etc/fstab (/root/a/etc/fstab outside the chroot) but mine was fine and didn't need editing. The next file is /etc/crypttab that needs to be created. In our example here it would be:
system /dev/sda3 none luks
You can use a UUID="840311bc-9333-47f6-b64f-a9becf3c5b1e" style entry there in place of /dev/sda3 And the next step that got me was to tell grub about this setup. vim /etc/defaults/grub (/root/a/etc/defaults/grub outside the chroot)
Change:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

to

GRUB_CMDLINE_LINUX_DEFAULT="cryptopts=target=system,source=/dev/sda3,lvm=luks quiet splash"
Change as required. Then run update-grub.

Done!


Posted by DaveQB | Permanent Link | Categories: IT
HERE!