Project 1999

Project 1999 (/forums/index.php)
-   Blue Trading Hub (/forums/forumdisplay.php?f=27)
-   -   Realtime auction logger (hourly refresh, may increase) (/forums/showthread.php?t=65187)

tristantio 02-18-2012 02:57 AM

Well guys, I am going to have PC shut down for a bit, hopefully someone else sends up some logs. If you're interested, below is my small shell script (Linux) for reading and sending the log contents from a toon parked in EC with /log turned on.

Code:

#!/bin/bash

LOGDIR=~/eqLogs
for f in $LOGDIR/eqlog*; do
  #Quickly check and make sure its been updated in last minute if we're going to send it
  echo "Checking $f"
  let now=$(date +%s)-60
  [ $(stat --printf=%Z $f) -lt $now ] && continue;
  echo "Working with: $f"
  curl -F "dump=$(tail -n100 $f)" http://ahungry.com/aucDump.php
done


DoucLangur 02-18-2012 10:28 AM

Quote:

Originally Posted by tristantio (Post 559362)
Code:

#!/bin/bash

Good choice! :D shell scripts ftw

tristantio 02-20-2012 03:52 AM

Hi all,

The C program is up on my site, you can view it from the main auction page.

I currently need to boot into windows to create some pre-compiled binaries (right now I just have the source code up).

Please contribute your logs - I also have a basic upload option if you don't feel like running a program to do uploads for you.

Few things - the C program could probably be written better (right now I am saving a portion of the large log file to an out file, which means you'll have to have write permission in the directory you run the program).

Swish 02-20-2012 10:56 AM

Enjoying this, using it more than I thought I would... will start getting logs in as well :)

phobus 02-20-2012 01:51 PM

Hey, so I tried running this under OS X and got some errors because the BSD version of stat(1) doesn't support --printf. Also, my log dir includes a space which was screwing with the for loop so I hacked the IFS variable. This works for me, and probably will under Linux too:

Code:

#!/bin/bash

# Define your log directory here - easiest to symlink to your wine DIR
LOGDIR="/Applications/Project\ 1999.app/drive_c/eq/Logs"

# Don't mess with this!
IFS=$'\n'

while :; do
        for f in `find ${LOGDIR}/eqlog* -newerct "1 minute ago"`; do
                echo "Working with: $f"
                curl -F "dump=$(tail -n 100 $f | grep auctions)" http://ahungry.com/aucDump.php
        done
        sleep 60
done


tristantio 02-20-2012 02:50 PM

Nice Phobus, works and much cleaner looking.

Writing IFS=$'\n' might be more legible but otherwise great change, I'll update as soon as I can on my site.

phobus 02-20-2012 04:45 PM

Ah, I knew there was a better way to do that but couldn't remember it. Thanks!

Just made some more changes: <strike>now it'll only grab the lines from the past minute including the word "auctions". So you won't get accused of spying on peoples' guild chat, and you won't get quite as much traffic from this</strike>. Aww, I can't seem to find an easy cross-platform way to do this. Oh well.

BTW, I never even know find would take a date specification like that. The manpage I was reading cryptically says it "is instead interpreted as a direct date specification of the form understood by cvs(1)" which I assumed meant some archaic hardcore Unix shit, but it turns out it's actually really flexible, which is cool.

Wizerud 02-20-2012 05:12 PM

Very nice work, thanks for this.

tristantio 02-20-2012 05:43 PM

I hit the same issue with cross platform parsing of just /auction text (which is why I opted to do it on the server end).

I could have either had windows users install strawberry perl at around 50M or create the C program which would be way smaller.

Damn Windows /shakes fist

tristantio 02-21-2012 12:11 AM

Btw, on Linux adding a grep -E will let you pre-parse so you don't send anything but auctions. I have also updated the C source to include this type of regex parsing.

I don't have a mac or bsd system set up right now to test it's grep functionality.

Code:

#!/bin/bash

#Define your log directory here - easiest to symlink to your wine DIR
LOGDIR=~/eqLogs

#Don't mess with this!
IFS=$'\n'

while :; do
  for f in `find ${LOGDIR}/eqlog* -newerct "1 minute ago"`; do
      echo "Working with: $f"
      curl -F "dump=$(tail -n100 $f|grep -E '[A-Za-z] auctions, ')" http://ahungry.com/aucDump.php
  done
  sleep 60
done



All times are GMT -4. The time now is 07:41 PM.

Powered by vBulletin®
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.