Realtime auction logger (hourly refresh, may increase)

[tristantio]

Well guys, I am going to have PC shut down for a bit, hopefully someone else sends up some logs. If you're interested, below is my small shell script (Linux) for reading and sending the log contents from a toon parked in EC with /log turned on.

#!/bin/bash

LOGDIR=~/eqLogs
for f in $LOGDIR/eqlog*; do
   #Quickly check and make sure its been updated in last minute if we're going to send it
   echo "Checking $f"
   let now=$(date +%s)-60
   [ $(stat --printf=%Z $f) -lt $now ] && continue;
   echo "Working with: $f"
   curl -F "dump=$(tail -n100 $f)" http://ahungry.com/aucDump.php
done

[DoucLangur]

Originally Posted by tristantio [You must be logged in to view images. Log in or Register.]

#!/bin/bash

Good choice! [You must be logged in to view images. Log in or Register.] shell scripts ftw

[tristantio]

Hi all,

The C program is up on my site, you can view it from the main auction page.

I currently need to boot into windows to create some pre-compiled binaries (right now I just have the source code up).

Please contribute your logs - I also have a basic upload option if you don't feel like running a program to do uploads for you.

Few things - the C program could probably be written better (right now I am saving a portion of the large log file to an out file, which means you'll have to have write permission in the directory you run the program).

[Swish]

Enjoying this, using it more than I thought I would... will start getting logs in as well [You must be logged in to view images. Log in or Register.]

[phobus]

Hey, so I tried running this under OS X and got some errors because the BSD version of stat(1) doesn't support --printf. Also, my log dir includes a space which was screwing with the for loop so I hacked the IFS variable. This works for me, and probably will under Linux too:

#!/bin/bash

# Define your log directory here - easiest to symlink to your wine DIR
LOGDIR="/Applications/Project\ 1999.app/drive_c/eq/Logs"

# Don't mess with this!
IFS=$'\n'

while :; do
        for f in `find ${LOGDIR}/eqlog* -newerct "1 minute ago"`; do
                echo "Working with: $f"
                curl -F "dump=$(tail -n 100 $f | grep auctions)" http://ahungry.com/aucDump.php
        done
        sleep 60
done

[tristantio]

Nice Phobus, works and much cleaner looking.

Writing IFS=$'\n' might be more legible but otherwise great change, I'll update as soon as I can on my site.

[phobus]

Ah, I knew there was a better way to do that but couldn't remember it. Thanks!

Just made some more changes: <strike>now it'll only grab the lines from the past minute including the word "auctions". So you won't get accused of spying on peoples' guild chat, and you won't get quite as much traffic from this</strike>. Aww, I can't seem to find an easy cross-platform way to do this. Oh well.

BTW, I never even know find would take a date specification like that. The manpage I was reading cryptically says it "is instead interpreted as a direct date specification of the form understood by cvs(1)" which I assumed meant some archaic hardcore Unix shit, but it turns out it's actually really flexible, which is cool.

[Wizerud]

Very nice work, thanks for this.

[tristantio]

I hit the same issue with cross platform parsing of just /auction text (which is why I opted to do it on the server end).

I could have either had windows users install strawberry perl at around 50M or create the C program which would be way smaller.

Damn Windows /shakes fist

[tristantio]

Btw, on Linux adding a grep -E will let you pre-parse so you don't send anything but auctions. I have also updated the C source to include this type of regex parsing.

I don't have a mac or bsd system set up right now to test it's grep functionality.

#!/bin/bash

#Define your log directory here - easiest to symlink to your wine DIR
LOGDIR=~/eqLogs

#Don't mess with this!
IFS=$'\n'

while :; do
   for f in `find ${LOGDIR}/eqlog* -newerct "1 minute ago"`; do
      echo "Working with: $f"
      curl -F "dump=$(tail -n100 $f|grep -E '[A-Za-z] auctions, ')" http://ahungry.com/aucDump.php
   done
   sleep 60
done