Archive for December, 2008

Hiawatha: a secure and advanced webserver

Hiawatha is a secure, full-featured and lightweight web server, licensed under GPL 2. Features are:

  • Access/connection control
  • Banning
  • CGI support (including PHP)
  • CGI wrapper (run CGI under a diffrerent uid/gid in a secure way)
  • Chunked Transfer-encoding
  • CommandChannel (control Hiawatha by telnetting to a special port)
  • Cookie support
  • Cross-site scripting prevention
  • Customizable ServerString
  • Directory listing with customizable stylesheet
  • DoS/flooding protection
  • FastCGI support (load-balanced)
  • GZip Content-Encoding support
  • HTTP authentication (basic and digest)
  • If-(Un)Modified-Since header field
  • Interface binding control
  • Internal file caching
  • IPv6 support
  • Keep-alive connections
  • Large file support
  • Logging
  • Request pipelining
  • Range header field (single range support)
  • Referer control for images (prevent external image linking)
  • Rootjail
  • SSL support
  • SQL injection prevention
  • Traffic throttling/shaping (control uploadspeed)
  • URL rewriting via regular expressions
  • Userdirectory support
  • Virtual host support
  • Volatile object support (for frequently changing objects, like webcam pictures)
  • XSLT support

Very interesting are the bolded security features and the build in XSLT support. It does not seem to be in the official repositories of the linux distributions.

Compile and run tests of Hiawatha have successfully been done (by me or others) on Debian, Ubuntu, Gentoo, Fedora, Slackware, FreeBSD, OpenBSD, NetBSD, MacOS X, Solaris and Cygwin. Because of the use of autoconf, it’s very likely that Hiawatha will compile and run on other Unix-clones as well.

A package/port is available for the following operating systems: ArchLinux, Debian, Fedora, FreeBSD, MacOS X, OpenBSD, SUSE and Windows.

Assign IPs to geographical Location

Assign IPs to geographical Location is quite interessting: if you know where visitors come from. you can redirect them to the web shop of their country, you can keep some countries away of your SSH daemon or showing special offers or events of this country.

Of course this is not bullet proof. If your visitors using proxies located in a different country, anonymizer like TOR or accessing the internet by VPN so GeoIP won’t work as expected.

Geo location of an IP is also known as GeoIP, because the only service most of the people know is named GeoIP by MaxMind.

GeoIP of MaxMind

GeoIP is a very popular service and often in open source software used. MaxMind, the company behind GeoIP provides a binary database monthly updated for free use (but is not open source). Most of the modern programming languages have APIs to this service. Try out the demo.

On modern Linux systems there is a package named geoip-bin:

 aptitude install geoip-bin

Download the binary database.

mkdir /usr/local/share/GeoIP
cd /usr/local/share/GeoIP
wget http://www.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
gunzip GeoLiteCity.dat.gz
ln -s /usr/local/share/GeoIP/GeoLiteCity.dat /usr/share/GeoIP/GeoIPCity.dat

use it

geoiplookup 84.226.106.171

NetOp – DNS based IP location service

DNS is always a good solutions if you want resolve a hostname or ip to something. NetOP.org provides such a DNS based location service. So the website says:

Existing approaches to IP geolocation chose to invent their own non-standard APIs. We feel that is not necessary. DNS is a highly efficient way to export IP-to- datasets.

This is so true.


Serving data via DNS, rather than a flat file, guarantees that answers to queries remain fresh.
….
With the country.netop.org service, NetOp demonstrates here that no special APIs are needed. All you need is a standard DNS resolver API.

NetOp runs a service that allows the public Internet to make DNSBL-style queries for a given IPv4 address’s ISO 3166 country code, as stored in a TXT RR

You can simply lookup a IP like this.

If you have a IP 84.226.106.171, make it reverse 171.106.226.84. And then use your preferred dns lookup tool:

dig 171.106.226.84.country.netop.org TXT

You will get a answer like

...
;; ANSWER SECTION:
171.106.226.84.country.netop.org. 604800 IN TXT	"CH"
...

As you can see, the IP is located in CH which stands for Switzerland. This service is not very detailed, it only shows countries for IP addresses. But very fast and easy to use.

With PHP this can be used like the following snippet on Linux/Unix. This snippet shows how you can use it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
<?php
// Nameserver we want to ask
$nameserver = 'country.netop.org';
 
// IP which we want to be check
// Normally use: $ip = $_SERVER['REMOTE_ADDR'];
$ip = "201.2.6.141"; // example IP
 
// Make IP reverse
$rev_ip = implode(array_reverse(explode('.', $ip)), '.');
 
// Default result
$country = "unknown";
 
// dns_get_record my not exist on some systems
if (is_callable("dns_get_record")) {
	$result = dns_get_record($rev_ip.'.'.$nameserver, TXT);
	if (!empty($result)) {
		// FIX: Not 100% sure if this works
		$country = $result[0];
	}
 
// dns_get_record is not available
} else {
	// we use dig on command line instead
	$command = 'dig '.$rev_ip.'.'.$nameserver.' TXT | grep ^[^\;]';
	$result = shell_exec($command);
 
	// Parse the dig output to only show the 2 letters of country code
	if (preg_match('/"([a-zA-Z]{2,2})"/',$result, $matches)) {
		$country = $matches[1];
	}
}
echo $country;

NetOp is still beta but works great for me. Hope you like it too.

HostIP

HostIP is an open project. See the examples which show how to use this service, the API uses HTTP GET. But you are also able to download the database (MYSQL DUMP ~25 MB GZIP) by HTTP or even better by RSYNC (MySQL, BDB, CSV) from mirrors.

Note: we are actively looking for rsync mirrors so please shoot us an email if you’re interested!

If you wish to contribute, there is a GIT repo to pull from and of course an e-mail address to send patches to. On this site, you can also find some API Extensions, Firefox extensions, etc.

So, time for a quick example. To only show a flag of the country of your visitors, it is simple as:

1
2
3
<a href="http://www.hostip.info">
<img src="http://api.hostip.info/flag.php" alt="IP Address Lookup" />
</a>

InfoSniper

Another nice service I found is infosniper. But consider:

We offer you 15 location queries per day for free. Interested in using this service commercially? We charge 10 EUR for 50,000 successful location queries. So a single location query costs you just 0.0002 EUR.

They provide a lot of scripts, API access examples and gadgets on their website. The IP geolocation query also does a whois query too go get more information.

Have phun.

Cherokee: very fast webserver

cherokee

Cherokee [1] (licensed under the GNU General Public License, version 2) is a very fast [2], flexible and easy to configure Web Server. It supports the widespread technologies nowadays: FastCGI, SCGI, PHP, CGI, SSI, TLS and SSL encrypted connections, Virtual hosts, Authentication, on the fly encoding, Load Balancing, Apache compatible log files, Data Base Balancer, Reverse HTTP Proxy and much more.

A user friendly interface called cherokee-admin [3] is provided for a no-hassle configuration of the server. Check out the benchmarks and documentation to learn more, and give it a try to squeeze your hardware to the fullest!

…and…

Cherokee is very light, completely modular and it can be tailored to your specific needs. As such, disk requirements will vary depending on the options selected for the building process. A static build for embedded devices can occupy as little as 200KB, give or take.

In terms of processing power, it has been known to work with as little as 133Mhz ARM processors. It could very well work on something smaller, but we haven’t really had the chance to try it out.

[1] http://www.cherokee-project.com/
[2] http://www.cherokee-project.com/benchmarks.html
[3] http://www.cherokee-project.com/doc/bundle_cherokee-admin.html

BotHunter: passive network monitoring tool

BotHunter [1] is a passive network monitoring tool designed to recognize the communication patterns of malware-infected computers within your network perimeter.  Using an advanced infection-dialog-based event correlation engine (patent pending), BotHunter represents the most in-depth network-based malware infection diagnosis system available today.

[1] http://www.bothunter.net/

AtMail: fresh looking webmail

AtMail [1] (Apache 2.0 Open Source license) is an open source webmail client written in PHP. We aim to provide a elegant Ajax webmail client for existing IMAP mailservers, with less bloat and a focus on an intuitive, simple user interface.

The open source version of AtMail provides users with a lightweight, yet powerful webmail client. The software can be installed on a variety of platforms with ease and without the hassles that most webmail platforms impart.

Traditional desktop mail clients are in decline; the future of email is via a web interface. AtMail is poised to deliver the next generation in Open Source Webmail.

Features:

  • Lightweight Ajax Webmail Interface
  • Video Mail
  • PHP source code
  • IMAP support
  • Live Spell Check
  • Address Book

atmail.org

There is also a full featured commercial version of AtMail webserver and/or webmail client. Look at the demos to see the difference between opensouce and commerical edition.

[1] http://atmail.org/

Rsnapshot: poor men’s timemachine

One of my favorite backup app is:

rsnapshot [1] is a filesystem snapshot utility for making backups of local and remote systems.

Using rsync and hard links, it is possible to keep multiple, full backups instantly available. The disk space required is just a little more than the space of one full backup, plus incrementals.

Depending on your configuration, it is quite possible to set up in just a few minutes. Files can be restored by the users who own them, without the root user getting involved.

There are no tapes to change, so once it’s set up, your backups can happen automatically untouched by human hands. And because rsnapshot only keeps a fixed (but configurable) number of snapshots, the amount of disk space used will not continuously grow.

rsnapshot is written entirely in Perl. It should work on any reasonably modern UNIX compatible OS, including: Debian GNU/Linux, Red Hat Linux, Fedora Linux, SuSE Linux, Gentoo Linux, Slackware Linux, FreeBSD, OpenBSD, NetBSD, Solaris, Mac OS X, and even IRIX.

Interesting is also the use of Rsnapshot accessing LVM.

[1] http://www.rsnapshot.org/

Nginx: high-performance web and proxy server.

Nginx [1] [2] is a high-performance web and proxy server. It has a lot of features, but it’s not for everyone. People generally use Nginx:

  • As an Apache replacement that gracefully handles many concurrent connections: Nginx is especially popular among web hosting providers. 50,000 simultaneous connections have been reported, thanks to Nginx’s use of epoll and kqueue.
  • As a load-balancing proxy server: Nginx is a popular front-end to Rails and PHP applications, either via FastCGI or HTTP. Written in C, it consumes about a quarter of the CPU that Perlbal uses.
  • As a mail proxy server: more of a niche application, but fastmail.fm reports great success.
  • As a server with a simple installation process, a clean configuration file, and few bugs: Nginx is easy to get running, and it almost never needs restarting. You can even upgrade the binary with zero downtime.

Nginx is very popular as you can see on Netcraft.

[1] http://nginx.net/
[2] http://wiki.codemongers.com/

paldo: Swiss GNU/Linux Distribution

I would like to here some feedback about this almost unknown GNU/Linux distribution:

paldo [1] is a Upkg [2] driven GNU/Linux distribution. It’s kind of a mix of a source and a binary distribution. Even though it builds packages like a source distribution it provides binary packages. paldo aims to be simple, pure, up to date and standards compliant.

[1] http://www.paldo.org/
[2] http://www.upkg.org/

Use DNS Blacklists in PHP

The following snippet is a quick and simple way to use dnsbl in PHP code (or for CLI use), if an IP is listed the code will return 1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<?php
/*
* Simple DNSBL check
* Author: Rene Moser
*/
// Check this IP
$ip = '201.2.6.141';
 
// List of DNSBL DNS Servers
$dns_black_lists = file('./dnsbl.txt', FILE_IGNORE_NEW_LINES);
 
// Reverse the IP
$rev_ip = implode(array_reverse(explode('.', $ip)), '.');
$response = array();
foreach ($dns_black_lists as $dns_black_list) {
	$response = (gethostbynamel($rev_ip . '.' . $dns_black_list));
	if (!empty($response)) {
		echo "1\n";
		exit;
	} 
}
echo "0\n";

The corresponding dnsbl.txt:

asiaspam.spamblocked.com
bl.deadbeef.com
bl.emailbasura.org
bl.spamcop.net
blackholes.five-ten-sg.com
blacklist.woody.ch
bogons.cymru.com
cbl.abuseat.org	cdl.anti-spam.org.cn
combined.abuse.ch
combined.rbl.msrbl.net
db.wpbl.info
dnsbl-1.uceprotect.net
dnsbl-2.uceprotect.net
dnsbl-3.uceprotect.net
dnsbl.abuse.ch
dnsbl.ahbl.org
dnsbl.cyberlogic.net
dnsbl.inps.de
dnsbl.njabl.org
dnsbl.sorbs.net
drone.abuse.ch
duinv.aupads.org
dul.dnsbl.sorbs.net
dul.ru
dyna.spamrats.com
dynip.rothen.com
eurospam.spamblocked.com
fl.chickenboner.biz
http.dnsbl.sorbs.net
images.rbl.msrbl.net
ips.backscatterer.org
isps.spamblocked.com
ix.dnsbl.manitu.net
korea.services.net
lacnic.spamblocked.com
misc.dnsbl.sorbs.net
noptr.spamrats.com
ohps.dnsbl.net.au
omrs.dnsbl.net.au
orvedb.aupads.org
osps.dnsbl.net.au
osrs.dnsbl.net.au
owfs.dnsbl.net.au
owps.dnsbl.net.au
pbl.spamhaus.org
phishing.rbl.msrbl.net
probes.dnsbl.net.au
proxy.bl.gweep.ca
proxy.block.transip.nl
psbl.surriel.com
rbl.interserver.net
rdts.dnsbl.net.au
relays.bl.gweep.ca
relays.bl.kundenserver.de
relays.nether.net
residential.block.transip.nl
ricn.dnsbl.net.au
rmst.dnsbl.net.au
sbl.spamhaus.org
short.rbl.jp
smtp.dnsbl.sorbs.net
socks.dnsbl.sorbs.net
spam.dnsbl.sorbs.net
spam.rbl.msrbl.net
spam.spamrats.com
spamlist.or.kr
spamrbl.imp.ch
t3direct.dnsbl.net.au
tor.ahbl.org
tor.dnsbl.sectoor.de
torserver.tor.dnsbl.sectoor.de
ubl.lashback.com
ubl.unsubscore.com
virbl.bit.nl
virus.rbl.jp
virus.rbl.msrbl.net
web.dnsbl.sorbs.net
wormrbl.imp.ch
xbl.spamhaus.org
zen.spamhaus.org

GnuPG Keysigning Party at ETH

Don’t miss the GnuPG Keysigning Party, Friday December 12th 7p.m. at ETH (HG E21). For more information see the official announcment.

Next Page »