i thought i should share it with you guys.
it's basically shit code that might still be bugged and weird because i just quick coded it without putting a lot of thought in it but it works most of the time.
you can download it here (make sure to click on the right download button - those upload sites are really confusing): http://www.file-upload.net/download-1219...b.jar.html
how does it work?
it's basically just a server and a bunch of crawlers. the program covers the following sites atm:
-http://lookatmyhorsemyhorseisamazing.com/ (<- wtf this is meant to be the fringe )
(i could add any site if someone wants me to do it)
you need an up-to-date java runtime environment ("jre" --- preferably oracles jre) to run it because it has been written in java.
save the jar-file into a folder and then run the following command prompt on it
java -jar TheWooHub.jar [SCAN INTERVAL]
sudo java -jar TheWooHub.jar [SCAN INTERVAL]
(you have to run it on linux as super user because the server binds on port 80 on your machine)
[SCAN INTERVAL] is the time between scans in milliseconds.^^ i recommend setting that value to 600000.
intervals below 10 min (600000 ms) might cause some problems with certain sites...
once it runs you'll see that it creates a couple of files. it needs them to work correctly. the "latestthread.*"-files are just there to keep track of the latest thread-id. they tell the program which threads are new. the "threadlist.woohub" file is where the program saves new threads with their respective links to the site. so each time you view the hub in your browser it will present you the threads that have been saved into that file.
the index.html file can be customized as you please if you don't like the layout. just don't delete or mess around with the
part because it will just fuck everything up. the program basically replaces #LIST# with the threads.
the index.html file is set to auto-refresh every 60 seconds as far as i remember... that might be annoying to some people so if you want to stop that shit just delete the
<meta http-equiv="refresh" content="60">
part inside the file and save it.
if you want to refresh it at other rates replace the 60 with any other number you want.
okay so once it's up and running on your host machine (which can be any machine in your home network - i run it on my pi) all you need to do is to visit the hub in your browser.
for example if you run the program on the same machine you'll use to browse the hub you'll just have to type in "http://localhost/" to view the hub.
if you're running it on another machine - eg like a raspberry pi - just type in the hostname of that machine into your browser - in my case that would be "http://failboat-pi/".
it now should present you a site that looks a bit like this:
of course the site won't have any threads in it when you just started the program. it will add new threads when the crawler finds them.
by clicking on "bookmark" you'll bookmark all threads with the same hub-id. previous threads - if there are any will disappear from your view and you'll see only the bookmarked threads and new threads as they appear. to reset the bookmark click on "reset bookmark".
clicking on "show X" will set the maximum amount of shown threads to the respective number.
the crawlers shouldn't raise any suspicion on the sites they visit. they should look like normal browsers to the site owners... but i encountered some issues on the fringe... if you set your interval to less than 10 min it will throw weird exceptions. probably some sort of security measures on their server? i really dunno.
i hope you guys like it. i run it on my pi and use it every day.