DNS Conditional Forwarding – dnsmasq

Why would I want to use Conditional Forwarding?

In my case, my local dns server has entries for local hostnames such as m2n.ion.lan, mongo.ion.lan, and tux.ion.lan. If I am using the vpn dns, then these address lookups would fail. By using Conditional Forwarding I can do all lookups locally, except for ones that match the remote top level domain (example.local). Anything that matches example.local would be forwarded to the remote dns server.

Problem:

  1. Connect to remote vpn server and use local DNS server
  2. Ping server.remote.local (remote FQDN) – fail
  3. Ping server.ion.lan (local FQDN) – success

Of course the remote ping fails because the local DNS server knows nothing about the remote domain. If I was to configure my machine to use the remote DNS server the opposite would happen. I would be able to ping server.remote.local, but a ping to server.ion.lan would fail.

Solution: Use dnsmasq with conditional forwarding to forward *.work.local requests to the remote dns server.

1. Install dnsmasq using your local package manager

2. Edit /etc/dnsmasq.conf

# Tells dnsmasq to forward anything with the domain of remote.local to dns server 10.25.11.2
server=/remote.local/10.25.11.2

# Listen to requests only coming from the local machine
listen-address=127.0.0.1

# Do not cache anything
# A decent dns server will already cache for your local network
cache-size=0

3. Edit /etc/resolv.conf

# Local LAN Domain
domain ion.lan

# local dnsmasq server
nameserver 127.0.0.1

# Your main dns server (dnsmasq will forward all requests to this server)
nameserver 10.20.1.1

4. Start dnsmasq

5. Test – ping a local server and remote server using the FQDN

All dns requests will be forwarded to 10.20.1.1 except any matching *.remote.local. server.remote.local will be forwarded to 10.25.11.2



OpenVPN Client – DNS Script

The OpenVPN server can pass DNS servers and a domain name to the client. This gives the benefit of using the remote dns servers for local hostname lookups.

Finding a good script that worked to do this provide difficult…

In server.conf add:

push "dhcp-option DOMAIN ion.lan"
push "dhcp-option DNS 10.25.11.2"

Then save this script on the client in same location as the client config

#!/bin/bash

case "$1" in
    up)
	    mv /etc/resolv.conf /etc/resolv.conf.bak

		echo "# Generated by OpenVPN Client UP Script" > /etc/resolv.conf
		for opt in ${!foreign_option_*};
		do
	        echo ${!opt} | sed -e 's/dhcp-option DOMAIN/domain/g' -e 's/dhcp-option DNS/nameserver/g' >> /etc/resolv.conf
        done
        ;;
    down)
        mv /etc/resolv.conf.bak /etc/resolv.conf
        ;;
    *)
        echo "Pass either UP or DOWN"
        ;;
esac

In the client.conf add

script-security 2

up "./vpn_dns_update.sh up"
down "./vpn_dns_update.sh down"

Now connect and check /etc/resolv.conf to see if the VPN nameserver and domain is listed.



Syslinux

Syslinux is a simple bootloader for fat, ext2/3/4, and brtfs.

Syslinux works in the following way (in a nutshell):

  1. MBR looks for the active partition (the one tagged as bootable)
  2. The MBR loads the code found on the partition’s boot sector and executes it
  3. This code then loads the rest of the boot loader code from /boot partition (file: ldlinux.sys)
  4. COM32 modules are loaded to provide extra functionality such as a graphical menu or chain loading

For a more detailed explanation: https://wiki.archlinux.org/index.php/Syslinux#Syslinux_Boot_Process

Sample configuration:

UI vesamenu.c32
DEFAULT arch
PROMPT 0
MENU TITLE Boot Menu
MENU BACKGROUND splash.png
TIMEOUT 100

MENU WIDTH 78
MENU MARGIN 4
MENU ROWS 5
MENU VSHIFT 10
MENU TIMEOUTROW 13
MENU TABMSGROW 11
MENU CMDLINEROW 11
MENU HELPMSGROW 16
MENU HELPMSGENDROW 29

# Refer to http://syslinux.zytor.com/wiki/index.php/Doc/menu

MENU COLOR border       30;44   #40ffffff #a0000000 std
MENU COLOR title        1;36;44 #9033ccff #a0000000 std
MENU COLOR sel          7;37;40 #e0ffffff #20ffffff all
MENU COLOR unsel        37;44   #50ffffff #a0000000 std
MENU COLOR help         37;40   #c0ffffff #a0000000 std
MENU COLOR timeout_msg  37;40   #80ffffff #00000000 std
MENU COLOR timeout      1;37;40 #c0ffffff #00000000 std
MENU COLOR msg07        37;40   #90ffffff #a0000000 std
MENU COLOR tabmsg       31;40   #30ffffff #00000000 std

LABEL arch
	MENU LABEL Arch Linux
	LINUX /vmlinuz26
	APPEND root=/dev/sda2 ro nomodeset
	INITRD /kernel26.img

LABEL archfallback
	MENU LABEL Arch Linux Fallback
	LINUX /vmlinuz26
	APPEND root=/dev/sda2 ro nomodeset
	INITRD /kernel26-fallback.img

As you can see, the majority of the config contains MENU statements declaring colors and positing for the menu. If you removed all the MENU statements, the config would be less than 20 lines.

Screenshot:

Syslinux Graphical Boot Menu

I was lazy and took a screenshot of the Arch Linux installer menu. The configuration above generates the same menu, except there are only two boot choices, Arch Linux and Arch Linux Fallback.

The arch wiki has a whole lot of good information on configuring Syslinux. https://wiki.archlinux.org/index.php/Syslinux



iheartradio – command line (mplayer)

Well after some experimentation and playing around I have found some new information out regarding listening to iheartradio from the command line in Linux.

Newer versions of mplayer have support to play the rtmp:// protocol eliminating the need for rtmpdump.

Quick Recap on how to grab the rtmp:// url from an iheartradio stream

  • Go to http://p2.STATION_ID.ccomrcdn.com/player/player_dispatcher.html?section=radio&action=listen_live; where station id is the call letters (ex: wtfx-fm)
  • The attribute primary_location in <stream> contains the rtmp url

NOTE: The RTMP URL changes every 5-10 minutes! You must fetch the new url everytime.

To play the stream with mplayer:

mplayer "rtmp://cp21366.live.edgefcs.net/live/Lou_KY_WTFX-FM_OR..." -novideo

The -novideo option is very improtant otherwise mplayer will take 5+ minutes trying to find video for the stream (there is none).

This is all great, but this is a lot of work everytime you want to listen to a iheartradio stream. Therefore I have coded up a script.

The Script:

  • Asks you for the station id (ex: wtfx-fm)
  • Asks how long the stream should play for
  • Check for song information every 10 seconds
  • Not Perfect (Alpha Quality)
#!/usr/bin/env python

import subprocess
import time
import urllib2
import xml.etree.ElementTree as ET
import os
import datetime

#Location of Stream to be SAVED
DefaultStation="wtfx-fm"

def getXML():
    data=urllib2.urlopen('http://p2.'+station+'.ccomrcdn.com/player/player_dispatcher.html?section=radio&action=listen_live').read()
    xml=ET.fromstring(data)
    return xml

def getSongInfo():
    xml=getXML()
    artist=xml.find('ListenLiveInitialize/JustPlayed/song/artist').attrib['name']
    title=xml.find('ListenLiveInitialize/JustPlayed/song/track').attrib['track_title']
    return artist,title



while True:
    station=raw_input("Enter Station ID [" + DefaultStation + "]: ")

    if not station:
        station=DefaultStation
    try:
        xml=getXML()
    except urllib2.URLError:
        print "Error - Invalid Station ID or Web Server Problem - Try Again"
    else:
        break

while True:
    try:
        TIME=int(raw_input("Time Stream will Play (in minutes): "))
    except ValueError:
        print "Error - Invalid Time - Try Again"
    else:
        break

rtmpurl=xml.find("ListenLiveInitialize/StreamInfo/stream").attrib['primary_location']

mp=subprocess.Popen(['/usr/bin/mplayer', rtmpurl, '-novideo', '-ao', 'alsa', '-quiet'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
mpPID=mp.pid

endTime = datetime.datetime.now() + datetime.timedelta(minutes=TIME)

OldSongInfo=[]

while datetime.datetime.now() < endTime:
    SongInfo=getSongInfo()
    if not SongInfo == OldSongInfo:
        OldSongInfo = SongInfo
        print SongInfo[0] + " - " + SongInfo[1]
    time.sleep(5)



print("Stopping MPlayer...")
os.kill(mpPID, 2)

print "Done"

N900

I got this script working on the N900 by downloading the Maemo 5 SDK and compiling the mplayer binary from a recent svn snapshot. Luckily, I didn't run into any problems. Although mplayer works with the N900, without any patches, it is not flawless. For example, video  gets jittery and skips when the backlight switches off. I would recommend leaving mplayer form extra installed and storing the mplayer you compiled in /opt. With audio playback I  do not have any issues, especially with the iheartradio rtmp stream!



Listen to iheartradio without Flash

UPDATE:  http://pyther.net/blog/index.php/2010/08/iheartradio-command-line-mplayer/

If you have ever listened to any Clear Channel FM radio station then I am sure you have heard the ads to listen to the station online through iheartradio. The only problem is that iheartradio is a bulky and slow flash application. On a powerful desktop that isn’t a huge issue, but with my N900 (600mhz cpu, 256MB Ram) it takes over 5 minutes to start streaming the radio station. Of course, iheartradio has an application for the iPod and Blackberry, but no app for the N900.

I went on a quest to figure out how to listen to iheartradio without the bulky flash application and this is what I found.

Step 1:

The url of the a stations stream is can be found in a XML file, at URL “http://p2.STATION_NAME.ccomrcdn.com/player/player_dispatcher.html?section=radio&action=listen_live”

If I want to listen to The Fox (call letters: WTFX-FM), the URL of the XML would be “http://p2.wtfx-fm.ccomrcdn.com/player/player_dispatcher.html?section=radio&action=listen_live”

Open up the url in a web browser and grab the rtmp url which is between the <stream> tags. rtmp://cp21366.live.edgefcs.net/live/Lou_KY_WTFX-FM_OR@s7696?auth=daEcEbgdNb4a3bdcKdYcrcgcGara0c1c3cZ-bmC7wi-4q-LM3Y9_7nqEDps4CCulBtyp&aifp=1234&CHANNELID=981&CPROG=_&MARKET=LOUISVILLE-KY&REQUESTOR=WTFX-FM&SERVER_NAME=p2.wtfx-fm.ccomrcdn.com&SITE_ID=2038&STATION_ID=WTFX-FM&MNM=2&TYPEOFPLAY=0

Step 2:

Download and Install rtmpdump and mplayer

  • Use your distro’s repositories
  • rtmpdump is in AUR (archlinux)

Step 3:

Lastly open up the terminal and enter the following command: rtmpdump -r $RTMPURL -v | mplayer -

-r tells rtmpdump the url of the stream

-v tells rtmpdump that the stream is a live stream

The | (pipe) directs stdin to mplayer and the – after mplayer tells mplayer to read data from stdin

Example: rtmpdump -r "rtmp://cp21366.live.edgefcs.net/live/Lou_KY_WTFX-FM_OR@s7696?auth=daEcEbgdNb4a3bdcKdYcrcgcGara0c1c3cZ-bmC7wi-4q-LM3Y9_7nqEDps4CCulBtyp&aifp=1234&CHANNELID=981&CPROG=_&MARKET=LOUISVILLE-KY&REQUESTOR=WTFX-FM&SERVER_NAME=p2.wtfx-fm.ccomrcdn.com&SITE_ID=2038&STATION_ID=WTFX-FM&MNM=2&TYPEOFPLAY=0" -v | mplayer -

Things to watch out for:

  • The RTMP url may change/expire. I have not been able to confirm this yet, but if the RTMP url changes you will simply need to open the url of the xml file (step 1) and copy the new rtmp url

Sources:

Maybe when I get some more time and become more ambitious I will write a small python wrapper that will extract the url from the xml file and start the stream.



HDTV Recording (MPEG2) to DVD

I record an episode of Dateline NBC because one of my family members wanted to see it, but she wasn’t going to be home to catch the episode. After much fooling around I finally got something that worked well.

So I’m tossing this here in hopes I will remember it and that someone else will find it useful.

The recording was in HD at a resolution of 1920×1280.

mencoder -oac lavc -ovc lavc -of mpeg -mpegopts format=dvd -vf scale=720:480,harddup -srate 48000 -af lavcresample=48000 -lavcopts vcodec=mpeg2video:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=6000:keyint=18:trell:aspect=16/9:acodec=ac3:abitrate=224:threads=4 -ofps 30000/1001 -o output.mpg input.mpg

And as another note: When you have a 16/9 video and you scale it to 720×480 (4/3) it will show the full frame to the wide screen tv (making the video look correct) or it will be stretched on a fullscreen television if the dvd player is outputting 4/3 letterbox. If the dvd player is outputting in cropped mode it will cut the extra width of and just show you the 4/3 portion.

^^ – This is how I understand the processes. It may be extremely wrong. If it is and you have a better understanding please let me know!



The Boot Flag

Well it all began when I started to hear a buzzing noise coming from my server. At first I thought it was the fans, but I quickly found out it was the drive. So I figured I would copy everything to a “spare” drive I had around that was being used for backups. Well no matter what I would do I couldn’t get the motherboard to boot from this drive. Oddly, my eeepc and qemu where able to boot from the drive without a problem.

After six hours of trying to swap drives I gave up. The following weekend I remembered that the boot flag needs to be set on the boot partition to get the bios to boot the drive. Sure enough after setting the boot flag in cfdisk the bios booted the drive without a problem.

The BIOS should look for a mbr. If there is a mbr it should attempt to boot the drive. It shouldn’t care about the partition table at all! From what I know, the boot flag is used by the mbr to determine the boot partition. However, grub nor lilo (both linux boot loaders) care about this flag. The windows mbr is the only bootloader in which the flag matters.

My server has an Intel Little Falls board (D945GCLF) which is only about 2 years old so I was quite surprised when I realized the boot flag had to be set.



Use Keyboard to resume from standby

One of the things that would always irritate me, with Linux, was the fact that I could not resume my machine by hitting a key on the keyboard.  When I first searched for an answer, to this issue, many of the replies stated “look for an option in the BIOS.” To my dismay, I had no such option in the BIOS. After more searching I found /proc/acpi/wakeup!

/proc/acpi/wakeup looks like this:

Device	S-state	  Status   Sysfs node
UAR1	  S4	 disabled  pnp:00:08
SMB0	  S4	 disabled  pci:0000:00:01.1
USB0	  S4	 disabled  pci:0000:00:02.0
USB2	  S4	 disabled  pci:0000:00:02.1
US15	  S4	 disabled  pci:0000:00:04.0
US12	  S4	 disabled  pci:0000:00:04.1
NMAC	  S5	 disabled  pci:0000:00:0a.0
P0P1	  S4	 disabled  pci:0000:00:08.0
HDAC	  S4	 disabled
MXR0	  S4	 disabled  pci:0000:00:10.0
BR11	  S4	 disabled
BR12	  S4	 disabled  pci:0000:00:12.0
BR13	  S4	 disabled
BR14	  S4	 disabled
BR15	  S4	 disabled
BR16	  S4	 disabled
BR17	  S4	 disabled

Now this might be confusing, at first, but do not fear! We are interested in only two types of devices: USB and US

USB0	  S4	 disabled  pci:0000:00:02.0
USB2	  S4	 disabled  pci:0000:00:02.1
US15	  S4	 disabled  pci:0000:00:04.0
US12	  S4	 disabled  pci:0000:00:04.1

To figure out which device is which take the number after pci: and run grep on dmesg. Example for US15: dmesg | grep 0000:00:04.0

You will likely get a lot of output… you should look for something similar (Note: this differs by hardware, it likely won’t be the same)

[    6.164097] usb usb4: SerialNumber: 0000:00:04.0
[    7.284302] input: BTC USB Multimedia Keyboard as /devices/pci0000:00/0000:00:04.0/usb4/4-3/4-3:1.0/input/input2
[    7.284363] generic-usb 0003:046D:C312.0001: input,hidraw0: USB HID v1.10 Keyboard [BTC USB Multimedia Keyboard] on usb-0000:00:04.0-3/input0
[    7.300110] input: BTC USB Multimedia Keyboard as /devices/pci0000:00/0000:00:04.0/usb4/4-3/4-3:1.1/input/input3
[    7.300287] generic-usb 0003:046D:C312.0002: input,hiddev96,hidraw1: USB HID v1.10 Device [BTC USB Multimedia Keyboard] on usb-0000:00:04.0-3/input1

As you can see US15 is my USB keyboard so I will simply run echo "US15" > /proc/acpi/wakeup to allow US15 to wake up the computer.

USB0	  S4	 disabled  pci:0000:00:02.0
USB2	  S4	 disabled  pci:0000:00:02.1
US15	  S4	 enabled   pci:0000:00:04.0
US12	  S4	 disabled  pci:0000:00:04.1

If it is not appearant which devices are which there is always the trial and error process. Enable one, see if the desired device wakes up the machine and if it doesn’t, disable the device (by executing the echo command again) and try another.

Lastly just add the echo command to your startup script. On Arch /etc/rc.local is a good place.



Uploader

A few months ago, I was talking to a friend and he wanted to share some files with me. However, we had no easy of doing this. Pondering about the problem, I though it would be great if there was a uploader with some basic user authentication.  This brought about the birth of uploader. My biggest problem that I faced was the fact that I had no idea how to work with sessions and sqlite databases. That didn’t stop me though! During winter break and I went to tackle the challenge.

After digging through a lot of examples on the webpy.org website, I was eventually able to throw together something that worked. Winter break ended, the program was usable, and was left, without love, on my server.

Just last week, I started to play with uploader again and I noticed that the code was a disaster and confusing. So I started to rewrite the parts of the code that were bad and added comments. Since I already stumbled with sessions and database objects I had a much better understanding on how they worked and how to implement them into the code. During the last three days I have released uploader.

Github Page:  http://github.com/pyther/uploader
Project Page: http://pyther.net/projects/uploader/

Screenshots:

File UploadAdministrator Panel


Webkit

Webkit has been a piece of software I have been interested with for quite a while. For those who do not know, webkit is a web rendering engine. Safari and Google Chrome are two popular browsers that use webkit as their rendering engine.

Webkit was forked from the KHTML project by Apple in 2002. At the time it was know as WebCore and JavascriptCore.  A year later, Apple provided patches to the KHTML project. The KDE team was able to use some of the code, but much of it was poorly documented and the coding style differed greatly. In 2005, Apple opensourced their fork (WebCore and JavascriptCore) as webkit.

I first gave webkit a shot about a year or so ago. I read about midori, a web browser that uses webkit, and I decide to give it a try. Midori was very simple and basic, which I liked and it was faster than firefox. However, webkit would crash a lot when loading/rendering web pages, resulting in midori crashing. Of course nobody likes this and I switched back to Firefox. From that point on, I was impressed with what was going on and tried the combination of webkit/midori every few months.

Over the last year, there have been huge improvements with webkit.
Here is a look at the current status of webkit:

The Good:

  • Spell Check support. Let me explain. When spell check support was introduced it would detect misspelled word, but there was no way to see a list of suggestions or pick a suggestion. Before, when a suggestion was picked it would erase the word!  Now, it is possible fix the spelling mistake by selecting (highlighting) the word and right clicking. It is a tad bit annoying that the whole word needs be selected, but it something that you get use to.
  • Fast: If you thought Firefox was fast, webkit is even faster and it seems to do pretty decent job at rendering javascript quickly too.
  • HTML 5 Video: Youtube has deployed HTML 5 videos which only work in webkit based browsers at the moment. There are two file formats for html 5 video: OGG Theora and H264. Mozilla wants ogg theora to become the norm because it is an open and free format, whereas h264 is bound with licensing restrictions and the future is unknown for the codec (will it remain free?). Due to Mozilla stance on the issue and licensing issues, Firefox only works with OGG Theora video clips. Youtube videos are encoded in H264 therefore they won’t work with Firefox. Webkit gets around this issue by using the native operating system’s video player. In the case of linux, webkit uses gstreamer, which results in all videos being processed by gstreamer, not webkit.
  • Developer Tools: Although the inspect page/element program needs some work, it has some very promising features. It is similar to firebug, but it also has features that allow you to see what is being stored locally (cookie, session info, etc..), how fast your page is loading, how long each element takes to load, an error console, and a scripts debugger.
  • Independent: Webkit is independent from any browser. This makes it much easier to integrate webkit into a web browser. Google chrome, Safari, Midori, Aora, and Uzbl are just a few browsers that use webkit as their rendering backend. As a result, you get developers that will provide patches to Webkit because they want their browser to do x, y, z or render pages better.

The Bad:

  • Crashes: Although all web browser crash at some point in time webkit still seems to crash every now and then. Granted, within the last few days I’ve only had one or two crashes, but Firefox and other browsers rarely ever crash (unless if flash is loaded). However, Midori and other webkit browsers can restore the session if their is a crash.
  • API: There is still a lot that needs to be done with the API. For example, form auto complete needs to be implemented. Any form auto complete that is seen right now is being implement by the browser (midori, safari, chrome). It would also be nice to get right click spelling to work (without having to select the word). Lastly, the organization of the right click menus could be improved.
  • Webpages: There are a handle full of web pages that simply won’t work with webkit right now. The two that come to mind are the University of Akrons Zipline (Student Center, specifically) and Cisco Netacad. Both of these sites use long and complex URLs and the cisco netacad site is built around flash.

Webkit has seen major improvements, but still has a long way to go until it reaches the maturity and stability of other browser on the market. I would consider webkit to be usable on day to day basis providing you don’t have to access one of those overly complex pages that don’t work well with webkit.

Arch Linux AUR Packages:
midori-git: http://aur.archlinux.org/packages.php?ID=14349
libwebkit-nightly:  http://aur.archlinux.org/packages.php?ID=34814