So I'm looking for best practices on logging. IO, resource consumption, etc.

Which is the better way to go from a performance perspective. Log to database, or filesystem.

Some of the applications can log straight to the database, apache being one. Some would have to be funneled from the syslog aggregation system to the database.

So on the applications that can go directly to the database should I, or should I just keep the standard, and let the aggregate do the work of putting into the database.



asked 07 Dec '10, 17:37

rfelsburg's gravatar image

rfelsburg ♦
accept rate: 25%

hmmmm many ways to do it really. In the end, you want it in the database, but what are you going to do with the data after that? Is it just so you can look at the complete history of your servers, clients and network overall?

I guess if it were me doing it, I'd have everything go to a centralized logs server running a *software RAID 0+1 setup by using mdadm and 4 x 1TB drives (or larger drives) and then have the database exist on there, importing all data to it, then you could have one location to which all data goes and is read from. I'd also keep all resources to a minimum by not running anything that is absolutely NOT needed on that one-purpose-system.

One caveat is that Linux has issues with drives that are larger than 512-sectors. The 1024 are newer, and support will come, but it's not there yet. I believe larger drives are broken down into multiple 512-sectors to counter this, but it's a hack workaround at best.

(I say to use RAID 0+1 so you get both speed and redundancy. RAID 3 would offer speed, but no redundancy. RAID 5 and 6, (1 and 2 parity drives, respectively) have inherent read/write errors vs RAID 0+1. I say software vs hardware RAID because if the mobo dies and you have a hardware RAID, it's problematic. Software RAIDs are much more portable than their hardware counterparts are.)


answered 09 Dec '10, 22:54

Ron's gravatar image

Ron ♦
accept rate: 13%

I've essentially started with the standard centralized log server setup, I'm working with Octopussy [] at the moment, which has just been a joy of a log analyser. If you haven't looked at it I highly recommend it.

(10 Apr '11, 17:56) rfelsburg ♦

Thanks for the tip!! I'll check that out now.

(11 Apr '11, 10:00) Ron ♦
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text]( "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported



Asked: 07 Dec '10, 17:37

Seen: 1,846 times

Last updated: 11 Apr '11, 10:00

powered by OSQA