911 Timeline Moved to S3
I finished moving my mirror of Paul Thompson's 911 Timeline to S3. billstclair.com/911timeline now redirects to a page documenting the move. All other links to the timeline automatically redirect. Apache's Redirect and RedirectMatch directives at work.
Next step in my S3 Ruby library: streaming upload of files and web URLs and download of files. The current Amazon-provided library requires objects to fit in memory.
I've got streaming upload
I've got streaming upload working in Ruby, but I haven't cleaned up the code for release. It works for what I need, uploading files to my S3 web sites. I have the following command line commands working well enough for limited use:
s3cd - change s3 working directory
s3pwd - print s3 working directory
s3ls - list files matching a prefix
s3rm - remove a file from s3
s3cp - upload local files to s3
My plan is for s3ls and s3rm to do full wildcard matching, and for s3cp to copy from s3 to disk and to copy from an http or ftp connection to s3 with no disk in between. This last isn't possible in general, since s3 requires that you know the object length at the beginning of the upload, but it will work for connections that give their length right away. Others need to be downloaded to disk first.
s3cp shows progress, like wget and scp. This is how I know that my streaming code is working; it shows upload progress once a second.
But I haven't done any work on it in a while, having enough to do what I need.
Previous Posts:
"A" for Vendetta
Quote
So THIS is the busybody nitwit!
Quote
MailCrypt: PGP and GNUPG for Emacs
Medical marijuana champion back in jail
No News For Saint Patty's
S3 Says "Hello World"
How Hardyville Fell Off the Map
New Poll Finds 86 Percent Of Americans Don't Want To Have A Country Anymore
Streaming to S3
I've got streaming upload working (I think) by using the code posted here: http://blogs.missiondata.com/?p=29. I say I think, because I've no idea how to test if streaming is working. Any ideas?
The next step is to add an update progress bar, but that's looking rather complicated. I'd be interested to know how you are progressing.
Edit comment