Project Idorus

Idorus is a wrapper around wget, curl (other http downloaders can easily be added) to download multiple files in parallel; it simply invokes the chosen downloader (up to a configurable number of simultaneous sessions)for each passed URL with the provided command-line arguments.

This is a pre-release, some parts are pretty rough but the basic functionality is there.

By default wget will be used (will be configurable in the future), a java property can be set to use curl (or a custom downloader).

By design, Idorus takes no command-line arguments (everything in the command-line will be passed verbatim in each downloader invocation), any configurable options can be defined from the command-line by use of Java properties, though.

Currently recognised properties are:

  • backend: Chooses with downloader to use
  • threads: Sets the maximum number of files to download simultaneously (when one of them completes a new one is started if some are remaining)
  • <backendname>_path: (Not really needed anymore, will ask automatically —only once— if required) Sets the path of the downloader executable.

Note  Idorus is built on top of pluggablejs so it they both must be downloaded; to ease usage I'm also publishing a jar file containing both.

For these examples I'll be using the full Java command-line, a wrapper will be provided in future releases.

  • Default settings (use wget and 4 simultaneous downloads)
    $ java -jar idorus.jar http://someurl1 http://someurl2 http://someurl3 http://someurl4 http://someurl5
  • Use curl and download two files simultaneously at most
    $ java -Dbackend=curl -Dthreads=2 -jar idorus.jar http://someurl1 http://someurl2 http://someurl3 http://someurl4 http://someurl5
  • Get a list of backends
    $ java -Dbackend=? -jar idorus.jar
  • Get a list of backends and the plugin file from where each is loaded
    $ java -Dbackend?? -jar idorus.jar

This is a pre-release, expect some unpolished edges.


© 2008 Toni Corvera <outlyer at gmail dot com>

All dates/times in this page are UTC.
  • idorus.txt
  • Last modified: 2016/04/25 01:20
  • by Toni Corvera