People were leeching off it and so hcs took it down. =(
Shame really, as there was quite a bit I hadn't downloaded yet... (I was only downloading one at a time)
edited 3:23 PM EDT August 4, 2008
Well crap by JackyF at 5:48 PM EDT on August 4, 2008
Thanks for the response. Man that really sucks, and I'm sorry to hear that. So how is everyone here on the boards getting the old and new files then? Could we implement a member system with names/passwords?
It just isn't worth the trouble of the site going down. It was a poorly organized collection of bad rips anyway. If you want to suggest a fully defined method for making it available I'll be glad to present a counter-argument.
I think what hcs is asking is how specifically to distinguish "regular users" from anyone else. Is it someone with 10 or more posts who's posted at least once in the last 7 days? Someone with 50 posts or more? Someone who's simply registered? "Regular users" is pretty vague. I could be misinterpreting, though.
@JILost: Yes, you hit one of the points. The other point is how to get the login info to the "regular users". Do these people send me email? Which I then have to check against the forum to see if they are active? How do I know who is who? And when something inevitably goes wrong and bandwidth usage becomes excessive what do I do?
@Knurek: An FTP would be nice, but it would be better for someone else to do it. I don't have the upstream bandwidth locally for it to be reasonable, and servage doesn't do that sort of thing.
The bonus folder was handy when you didn't have to worry about ratios, or if you wanted a "quick" way to find certain sets. It was there, and it was reliable most of the time. But I fully agree with the decision, the page/board was (IS) more important to keep up than the sets themselves (really).
Proposing alternativies? Nah, been a few years since I grabbed streams, they are usually huge for my hdd so I'd rather not bother at the time. Not to mention that the newer the games the strangest music they came out with, like it's not as 'bleepy' as it used to be :( (I'm willing to listen to recomendations).
Maybe this will turn out like a release by week site? (gamemp3s like) where streams are put on public tracker for people to leech? That could work, but also totally fail due to people not seeding, or waiting days for that final 0.01% for the file in question to be finished.
In any case, thanks for you hard work, guys. ps: this is not a problem, just a dilemma as to how to make sets available for everyone else.
edited 9:00 AM EDT August 5, 2008 > yay! for opening spanish question marks
"Maybe this will turn out like a release by week site? (gamemp3s like) where streams are put on public tracker for people to leech? That could work, but also totally fail due to people not seeding, or waiting days for that final 0.01% for the file in question to be finished."
You obviously weren't here when we had the torrent tracker. You were lucky if you could find a seed much less download a set in less than two days. Sure it is a good idea, but it is just doesn't work here.
It's true, torrents work best when a bunch of people are trying to get the same thing, not when a bunch of people are trying to get a bunch of things. The popular sets (Smash Bros Brawl, Super Mario Galaxy, generally new stuff when it comes out) are well served by torrents, but for most sets dedicated seeders would be needed.
Consider that the current "known universe" of streams is about 250 GB (zipped), from over one thousand games.
@anu: There's no question that /bonus was handy, and I enjoyed being able to provide it. It is with reluctance that I take it down, and that's why it has a tendency to come back... but as we saw earlier today even an unpublicized mini-bonus can cause trouble.
So, why did we go down today? I am beginning to think that there is a conspiracy to keep this site down. Is there a way for you to just ban some IPs and be over it?
@hcs: It's a strange problem. I guess we have become too popular for our own good. It is interesting though that if you Google "hcs64 bonus" you get the so-called secret as the first result.
Anyways, good luck in getting the issue sorted out.
I'm not aware of how the torrent system was set up previously, but it almost sounds like there was a tracker set up specifically for this purpose, maybe even a private one.
I suggest that a system is put in place that embodies what BitTorrent was originally designed to do, reduce the bandwidth used by public servers serving large files.
A torrent could be made for each release using those recommended by Mininova and possibly The Pirate Bay's trackers, then uploaded to multiple popular torrent sites and the torrent files or torrent pages could be linked to from hcs64.com.
The necessary permanent seeders could then be the servers that used to or currently host the files with BitTorrent clients set to have rate-limiting or bandwidth quotas set. With a proper rate-cap, even hcs64.com could be the only seeder and still keep within any bandwidth limits needed, but hopefully other mirrors will hop in and seed.
Currently there are two still active torrents for Wii ripped content that I've found (One and Two) that are mostly seeded, so there definitely is a demand for this stuff. Getting it on popular tracker sites increases the chances of people joining in to help.
The problem is though, you need dedicated seeders to begin with and you need decent peers who will retain seeder status once they finished downloading the data. If you didnt know, there most likely aren't many people willing to dedicate there bandwith to a particular collection of songs for leechers.
Long answer: No. It's always been on the site, but HCS had cut off all access to it. He would prefer that the sets be distributed in a better form than from his website (due to bandwidth issues). In case you're wondering, no, I don't have access, and neither does anyone else (except of course HCS). Mouser X over and out.
Wow. The music is taken down and all of these people with low post counts appear out of nowhere and beg for it to be put back up. No wonder the site stays up for only a few hours every time hcs puts up the streams...
As in, shunt off about 10 times the bandwidth to OTHER sites in order to get the ISO's, and do what's already done. Sorry, that's dumb.
I've already put up my idea, hcs should just start asking around for a place to host his site elsewhere if he wants to keep this place as the file host. Looking at it, I honestly think this place is pretty damn good seeing as you go over the limit, and they only take the site away for a few hours instead of the rest of the month, so as far as hosting the files goes, this place is pretty damn good about it.
Just get your site elsewhere, and if you want, take the smaller vg musics there as well (as in, the N64 and DS stuff). You can get set up at the new place, and change your dns redirect to the new place, and it'll only take about a day for it to spread across to other dns servers, and be like you never went anywhere, everything will be the same, just that your site will be separate from these files.
What's more, you can get better control over your site and do more with installed modules that servage doesn't provide (perl, for instance), and set up a better system to control access to stuff.
EDIT: Here's one idea you can enact right now until you get a more permanent solution... Have all your sets up, and deny access to them, only giving access to certain ones you are personally asked for. Then let them get it. I dunno. It would give you complete control over who's getting what, and can tell if there was more than 1 download if you can tell how much bandwidth was used.
Maybe make it work by setting up a certain folder with a password you can change whenever, and when you give someone access to get it, copy it to that folder to let them get it, and then delete it afterwards. Only problem is this would be very time consuming and you'd have to do a lot of pass changes, but if you feel like only letting a few people get anything right now, you can do that as a temporary solution.
See? Uf isn't that smart. A torrent seems like a good idea but not the best due to no one willing to seed it. Megaupload is feasible but hcs will need a premium account for that...
>> I've already put up my idea, hcs should just start asking around for a place to host his site elsewhere if he wants to keep this place as the file host.
Note the bolded text. Maybe hcs does not want to do that?
>> ...and they only take the site away for a few hours instead of the rest of the month...
I think that hcs has to pay in order to get the site back up. Maybe I am wrong about that, but I seem to remember this being the case in the past.
>> Just get your site elsewhere, and if you want, take the smaller vg musics there as well
Why don't you do it? You seem to be a willing volunteer with all of the answers... :P
>> <Insert DChronos's other idea which uses passwords>
I think that hcs wants a more hands-off solution. That is too involved. I say that hcs should perma-ban anyone who leeches the server offline if he is not doing so already.
@nenson: Be nice. We don't need a flame war in this topic.
I don't have to pay to get the site back, it is just a pain to wait. I really ought to move to another server, but I'd like to do that with everything, not just the forums. Because I'm sick and tired of servage, but too damned lazy to do anything about it. I don't like the idea of the free site DC has suggested.
Any recommendations of pay hosts? Money isn't the ultimate object, though it is important. Most important is quality. Any with a known excellent reputation?
http://www.leaseweb.com/ - Has hosted a lot of the famous torrent trackers, they've been known to crack down on really big sites but I'm part of several smaller trackers hosted on them and they're running fine with no end in sight.
http://www.ovh.co.uk/ - Well known on torrent forums for being a good place to get a seedbox and their nice bandwidth.
http://www.midphase.com/ - Good pricing and plans but I'm not sure how they feel about copyrighted stuff (In this same vein, Dreamhost would be awesome if they weren't so into taking issue with copyrighted stuff).
So basically anywhere that advertises their service as a potential seedbox should be a-okay in terms of copyright stuff (one example).
If anyone knows of a good torrent site on a reliable host but don't know who hosts them, this tool has worked for me.
Well regardless of whether the links are public or private, having a torrent client serve up the files instead of http links means that whoever is downloading will contribute help seed (even if it's just for the time their download takes) and reduce the load on the seeder(s) and who knows, some seeders might stick around for a while to help out with their favorite sets. So the choice of using torrents or not, regardless of anything else, is an obvious one.
I highly recommend submitting to the top trackers and indexing sites. As long as it has just one seed (read: never is shown as "dead") it will stay readily accessible to anyone looking for such data.
Personally, having Azureus open in the background doesn't harm anything here, so I wouldn't mind seeding on and off, but I'd of course have to have the sets first... well, that is, someone has to start.
As far as a pay host goes, it doesn't matter what pay host you get, if you got people DLing tens of GB of data a day, you'll be taken out. I say, if you have the money to do so, make a server and host yourself.
Another option could be... though I don't know if they will watch your bandwidth or not, but probably, a place that you can rent a server from, and you can rent an entire server to yourself. Check Dreamhost to read about what I mean, just one place that comes to mind that I remember doing that virtual host thing. You're free to do whatever, but I don't know how anywhere else will be about bandwidth and what you put up.
Also, the free site was just for you 1Mb of you forum and web pages so those don't go out when the file host does, there's absolutely no reason to feel guilty about getting that hosted for free. If they couldn't handle it, they wouldn't be running. If they didn't want you to be hosted for free, they wouldn't allow you to be.
Also, an FYI, I know how bad that password thing was, that's why I said myself it would be a sucky time wasting thing, and was just suggesting it as I wasn't leaving anything out.
One last thing, I had been chatting with hcs and he said he was thinking about separating the files and the site so the site stays up. As far as setting up a site elsewhere goes, it would be as easy as zipping all the files from here from the root directory, getting the database files, and putting them up elsewhere in 1 or 2 uploads, it isn't like it would be hard if I WAS doing it, marioman.
Even if I have my own server, I have to pay for bandwidth. Buying full access to all the bandwidth a box has is probably going to be out of my price range. A rate limited setup might be usable. If I have a system that I have more control over, I can run a torrent client there, or a better-controlled HTTP system of some sort.
I'd rather not go with the free site, thanks again but please stop suggesting it. Just as there are things I'd rather not pay for there are some things I rather would pay for. It's more of a mental problem than a rational decision.
Also, I really don't have the option of building a 100+ GB zip file on the server. I don't think preserving the current mishmash of old sets that /bonus had is worthwhile anyway. When I get something up I will use the more organized collection that manakoAT and friends have been so diligent in preparing.
As long as nothing is lost and the organized collection has everything that was in the previous bonus directory, but is just reorganized, it sounds fine.
When I was hunting around for hosting company recommendations, the suggestion of putting one's own server into colocation with a company like ColoPronto was brought up, much like what DChronos is suggesting. A site that hosts some pretty racy stuff is on there and either ColoPronto isn't aware of what's going on or doesn't care. Might be something to look into.
I'd say figure out how much bandwidth the main site takes per month and then add on how much you want to devote to seeding, with a bit of headroom. If the entire set of torrents is around 250GB then getting it out capped at 100KB/s would mean one full set will be done each 30 days, but with other seeders and leechers the time should be much shorter.
Please use TorrentZips and post a filelist with SFVs (CRC32s) and/or MD5s of what's inside each TorrentZip archive so jumping in to help seed is simple.
Mm, I don't think I want to go with full colocation, in the sense of just renting rack space. I'd rather have someone else set that up and handle problems on-site and, importantly, run backups.
The sets are all TorrentZipped, and manakoAT has been maintaining clrmamepro dats, so your file list demands are already met.
hcs could copy five links to the second bonus directory and change them every seven days. that way people wouldn't dl as much, but people still could access it. cmon, whats the password? you can tell me...
Obviously he can't, otherwise you wouldn't have had to ask for it. Please stop talking about it, as HCS is entirely aware of the situation that's going on, and you are entirely unaware. Mouser X over and out.
I have a different idea that I wonder if it could be useable, and I'm certain there is a way to do this...
What about getting a new file host, if you don't want to be here, in which you can cut off any access to files via direct links of any sort, though can give the option to certain people you may want to get a set to immediately in the dev team, and set up a tracker that can seed the files from the full time 24/7 host.
With this option, there would be both a place for your files for immediate access to certain people, and a full time 24/7 seeder for torrents of the files so you wouldn't have to worry about if anyone is seeding so someone can get it. It's the best of both worlds there, people can get it via torrent, and can get it from others that are DLing it, as well as the server if there isn't anyone seeding.
I'll look this up, but I don't see why this wouldn't be a feasible thing to accomplish. It would cut your mass downloading by a shitload, and cut up the bandwidth load between anyone else trying to get files, but still let you give direct access to the files to those you want to have it, hell, maybe even special members of your site that earn the privilege.
by nensondubois at 11:04 PM EDT on August 18, 2008
Your general idea of having torrents to distribute the files has been brought up and agreed on as the best solution. If you read a few pages back you'll see a new host is being looked into that is more friendly to running a torrent client for seeding. A new host might not even be necessary if TorrentFlux is able to run on the current server.
You seem to really be focusing on keeping things private though and there really isn't a need. The simple reality of a torrent going faster the more people that are on it makes trying to restrict access an utterly self-defeating action.
All the infrastructure and tools needed to maintain a torrent already exist, the only thing we need to add are seeders. The Pirate Bay and Mininova each are ideal places for using their recommended trackers and having them index the torrents so more people are likely to find them.
@nenson and wolupgm6: Here is a little advice - incessant begging for the login to the bonus directory can cause people to make false assumptions as to who is leeching the site offline. Secondly, do you really think that UF knows the password? Hcs said that he did not give it to anybody. Thirdly, if hcs gave only you the login and the site went offline, you would be in huge trouble. Why would either of you want to put yourself in that position?
Perhaps now is a good time to start then? After all, why not? Once you get a torrent client, it's just as easy as downloading off a website. And considering how stable Servage is, torrents are better. Servage frequently cuts people off while they're downloading (or sometimes uploading even) large files. Because of the way torrents work, if you get cut off during the download, no problem. In fact, it's pretty much part of the protocol. The only other thing I can think of against torrents is that, due to the way they're downloaded, you usually have to wait for the entire file to finish downloading before you can use it (whereas if you downloaded it from a website, it's usually downloaded sequentially, and thus you can use the first part of the file, even though the end of the file may be incomplete. But I don't see that as a problem, because it's rare that you weren't intending to download the entire file anyway).
Anyway, what's wrong with torrents? They work fine for me. Mouser X over.
Torrents rock.. As long as people seed when they are done its a good system to distribute stuff and less pressure on his bonus section :) I have had that cut off problem a few times.. But we can't complain when we are downloading great stuff :) ^_^
bonus used to be awesome... now its not... is it just me or is something wrong with this situation? and before anyone blames me for it coming down, i'd like to say that in a whole year, i only downloaded 8-12 files. thats roughly one a month.
anyway the point of this post was to state the fact that torrents work on 80% of the population's computers. the other 20% either have firewall or security issues; corrupt files managing their ports; or some other crazy error, stopping them from using torrents.
i do, however, have another suggestion: every archive that hcs has should be listed in a way, similar to the streamDB; then, you can post a "request", similar to the usf request list, except username is included. hcs then reviews this list, and if the user has done enough, hcs (or someone else) could upload the requested set at megaupload (or something of similar workings) in a temporary account. the file would only stay there for a month (without dls). this would save hcs having to go premium or; upload all sets, then when one goes down, reupload.
Thanks for the suggestion wolupgm6. No one's blaming you in particular.
bonus was awesome as long as not too many people knew about it. Bittorrent, while not supported as widely as HTTP, can be awesome all the time, and indeed gets more awesome as it gets more popular.
Regarding the requests system, it is indeed plausible, and I do fill requests from time to time. But to do really make things work (a content management system, a vast collection of auto bittorrent seeds, etc) requires an investment of time that I can't commit myself to spend. I'm working on vgmstream in what spare moments I have, which I think we can agree is a better use of my particular skills.
I think the best solution right now is to do what I've been doing, namely let someone else figure it out. If someone wants to set up a server for streams I can start throwing files at them. If someone has a good plan for how I could set this up that wouldn't require any involvement from me other than possibly funding and a seed data transfer I'd like to see the detailed proposal; either email me or put it in this thread. Or if someone wants to take it upon himself to fill requests via megaupload more power to you, feel free to use the forum for such stuff (as folks have been doing for years).
But I'm not going to commit myself to babysitting a request list, or put the site at risk of going over bandwidth again.
I understand where you are coming from, and, put in your position, I would do the exact same thing. As you probably know, I only post here once a week (Friday night is AUS) because the rest of my time is dedicated to programming and schooling. And I do know that noone is blaming me, I was just saying something before someone did.
Also, may I ask what Wii ISO explorer you used to get the Wii files? Just because I formatted my harddisk (again) the other day and lost various files.
@wolupgm6: Just to clarify, the comments that I made earlier were not blaming you or nenson for the site going offline. I was just pointing out that it looks really bad for someone to be asking for the password over and over when the site went down just a few weeks earlier.
Hcs told me to be quiet about the issue so I will, but I just would like to make sure that there are no hard feelings.
I managed to locate the Wii ISO explorer, but the common-key (key.bin) is hard to find when you cant use torrents. Could someone please nudge me in the right direction?
Chell, I am not talking about keeping torrents private and already knew torrents were being discussed.
I was talking about having the files hosted on a server that acts as a permanant seeder (and tracker, maybe) that's always up, so noone ever has to worry about if someone just happens to be on seeding it. A half Gb (that's about the average size of the files, though tons are bigger than that, even) torrent takes a hell of a lot of time to get from only a few people, and when most of those people even getting them don't stay on after they got what they want, torrents are defeated. At least if they go to torrents with this idea, there will always be a seed to get it from, even though it will take about a week to get it...
The private file idea was to allow only certain people to have direct access to the files on the server for instant downloading. Everyone else (the public) would be using torrents.
I still think the request thread going on has a better idea, though. The game music soundtracks thread at galbadia forums has hundreds of files up at megaupload, reuploaded by multiple people, and links changed when it needs to, but they stay up as long as there is activity. And 1 place, filefactory, keeps them for a long damn time.
I think in the end, the way it could go is to have different servers hosting some of the files. But for the sake of letting people get them, he should set up a seeder server/tracker. I already talked a bit with someone who knows a bit about it and knows it's possible, and have looked it up and found that it has been done already. There are a few programs I need to research a bit on when I get more time, that set this up and automatically make torrents and updates the page when you upload it to a files folder.