module Bio::Command

Bio::Command

Bio::Command is a collection of useful methods for execution of external commands or web applications. Any wrapper class for applications shall use this class.

Library internal use only. Users should not directly use it.

Constants

QUOTE_CHARS_WINDOWS
UNESCAPABLE_CHARS
UNSAFE_CHARS_UNIX

Public Instance Methods

call_command(cmd, options = {}) { |io| ... } click to toggle source

Executes the program. Automatically select popen for Ruby 1.9 or Windows environment and fork for the others. A block must be given. An IO object is passed to the block.

Available options:

:chdir => "path" : changes working directory to the specified path.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) options: Hash

Returns

(undefined)

# File lib/bio/command.rb, line 196
def call_command(cmd, options = {}, &block) #:yields: io
  if RUBY_VERSION >= "1.9.0" then
    return call_command_popen(cmd, options, &block)
  elsif no_fork? then
    call_command_popen(cmd, options, &block)
  else
    begin
      call_command_fork(cmd, options, &block)
    rescue NotImplementedError
      # fork(2) not implemented
      @@no_fork = true
      call_command_popen(cmd, options, &block)
    end
  end
end
call_command_fork(cmd, options = {}) { |io| ... } click to toggle source

This method is internally called from the #call_command method. In normal case, use #call_command, and do not call this method directly.

Executes the program via fork (by using IO.popen(“-”)) and exec. A block must be given. An IO object is passed to the block.

See the document of #call_command for available options.

Note for Ruby 1.8: In Ruby 1.8, from the view point of security, this method is recommended rather than call_command_popen. However, this method might have problems with multi-threads.

Note for Ruby 1.9: In Ruby 1.9, this method can not be used, because Thread.critical is removed. In Ruby 1.9, #call_command_popen is safe and robust enough, and is the recommended way, because IO.popen is improved to get a command-line as an array without calling shell.


Arguments:

  • (required) cmd: Array containing String objects

  • (optional) options: Hash

Returns

(undefined)

# File lib/bio/command.rb, line 373
def call_command_fork(cmd, options = {})
  dir = options[:chdir]
  cmd = safe_command_line_array(cmd)
  begin
  tc, Thread.critical, flag0, flag1 = Thread.critical, true, true, true
  IO.popen("-", "r+") do |io|
    if io then
      # parent
      flag0, Thread.critical, flag1 = false, tc, false
      yield io
    else
      # child
      Thread.critical = true # for safety, though already true
      GC.disable
      # chdir to options[:chdir] if available
      begin
        Dir.chdir(dir) if dir
      rescue Exception
        Process.exit!(1)
      end
      # executing the command
      begin
        Kernel.exec(*cmd)
      rescue Errno::ENOENT, Errno::EACCES
        Process.exit!(127)
      rescue Exception
      end
      Process.exit!(1)
    end
  end
  ensure
    # When IO.popen("-") raises error, Thread.critical will be set here.
    Thread.critical = tc if flag0 or flag1
    #warn 'Thread.critical might have wrong value.' if flag0 != flag1
  end
end
call_command_open3(cmd) { |pin, pout, perr| ... } click to toggle source

Executes the program via Open3.popen3 A block must be given. IO objects are passed to the block.

You would use this method only when you really need to get stderr.


Arguments:

  • (required) cmd: Array containing String objects

Returns

(undefined)

# File lib/bio/command.rb, line 419
def call_command_open3(cmd)
  cmd = safe_command_line_array(cmd)
  Open3.popen3(*cmd) do |pin, pout, perr|
    yield pin, pout, perr
  end
end
call_command_popen(cmd, options = {}, &block) click to toggle source

This method is internally called from the #call_command method. In normal case, use #call_command, and do not call this method directly.

Executes the program via IO.popen for OS which doesn't support fork. A block must be given. An IO object is passed to the block.

See the document of #call_command for available options.

Note for Ruby 1.8: In Ruby 1.8, although shell unsafe characters are escaped. If inescapable characters exists, it raises RuntimeError. So, #call_command_fork is normally recommended.

Note for Ruby 1.9: In Ruby 1.9, #call_command_popen is safe and robust enough, and is the recommended way, because IO.popen is improved to get a command-line as an array without calling shell.


Arguments:

  • (required) cmd: Array containing String objects

  • (optional) options: Hash

Returns

(undefined)

# File lib/bio/command.rb, line 235
def call_command_popen(cmd, options = {}, &block)
  if RUBY_VERSION >= "1.9.0" then
    if RUBY_ENGINE == 'jruby' then
      _call_command_popen_jruby19(cmd, options, &block)
    else
      _call_command_popen_ruby19(cmd, options, &block)
    end
  else
    _call_command_popen_ruby18(cmd, options, &block)
  end
end
escape_shell(str) click to toggle source

Escape special characters in command line string.


Arguments:

Returns

String object

# File lib/bio/command.rb, line 121
def escape_shell(str)
  if windows_platform? then
    escape_shell_windows(str)
  else
    escape_shell_unix(str)
  end
end
escape_shell_unix(str) click to toggle source

Escape special characters in command line string for UNIX shells.


Arguments:

Returns

String object

# File lib/bio/command.rb, line 110
def escape_shell_unix(str)
  str = str.to_s
  raise 'cannot escape control characters' if UNESCAPABLE_CHARS =~ str
  str.gsub(UNSAFE_CHARS_UNIX) { |x| "\\#{x}" }
end
escape_shell_windows(str) click to toggle source

Escape special characters in command line string for cmd.exe on Windows.


Arguments:

Returns

String object

# File lib/bio/command.rb, line 95
def escape_shell_windows(str)
  str = str.to_s
  raise 'cannot escape control characters' if UNESCAPABLE_CHARS =~ str
  if QUOTE_CHARS_WINDOWS =~ str then
    '"' + str.gsub(/\"/, '""') + '"'
  else
    String.new(str)
  end
end
http_post(http, path, data, header = {}) click to toggle source

Same as:

http = Net::HTTP.new(...); http.post(path, data, header)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Default Content-Type is application/octet-stream. Content-Length is automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.


Arguments:

  • (required) http: Net::HTTP object or compatible object

  • (required) path: String

  • (required) data: String containing data

  • (optional) header: Hash containing header strings

Returns

(same as Net::HTTP::post)

# File lib/bio/command.rb, line 940
def http_post(http, path, data, header = {})
  hash = {
    'Content-Type'   => 'application/octet-stream',
    'Content-Length' => data.length.to_s
  }
  hash.update(header)

  http.post(path, data, hash)
end
http_post_form(http, path, params = nil, header = {}) click to toggle source

Same as:

http = Net::HTTP.new(...); http.post_form(path, params)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Note that Content-Type and Content-Length are automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.


Arguments:

  • (required) http: Net::HTTP object or compatible object

  • (required) path: String

  • (optional) params: Hash containing parameters

  • (optional) header: Hash containing header strings

Returns

(same as Net::HTTP::post_form)

# File lib/bio/command.rb, line 812
def http_post_form(http, path, params = nil, header = {})
  data = make_cgi_params(params)

  hash = {
    'Content-Type'   => 'application/x-www-form-urlencoded',
    'Content-Length' => data.length.to_s
  }
  hash.update(header)

  http.post(path, data, hash)
end
make_cgi_params(params) click to toggle source

Builds parameter string for from Hash of parameters for application/x-www-form-urlencoded.


Arguments:

  • (required) params: Hash containing parameters

Returns

String

# File lib/bio/command.rb, line 866
def make_cgi_params(params)
  data = ""
  case params
  when Hash
    data = params.map do |key, val|
      make_cgi_params_key_value(key, val)
    end.join('&')
  when Array
    case params.first
    when Hash
      data = params.map do |hash|
        hash.map do |key, val|
          make_cgi_params_key_value(key, val)
        end
      end.join('&')
    when Array
      data = params.map do |key, val|
        make_cgi_params_key_value(key, val)
      end.join('&')
    when String
      data = params.map do |str|
        key, val = str.split(/\=/, 2)
        if val then
          make_cgi_params_key_value(key, val)
        else
          CGI.escape(str)
        end
      end.join('&')
    end
  when String
    raise TypeError, 'Bio::Command.make_cgi_params no longer accepts a single String as a form'
  end
  return data
end
make_cgi_params_key_value(key, value) click to toggle source

Builds parameter string for from a key string and a value (or values) for application/x-www-form-urlencoded.


Arguments:

Returns

String

# File lib/bio/command.rb, line 909
def make_cgi_params_key_value(key, value)
  result = []
  case value
  when Array
    value.each do |val|
      result << [key, val].map {|x| CGI.escape(x.to_s) }.join('=')
    end
  else
    result << [key, value].map {|x| CGI.escape(x.to_s) }.join('=')
  end
  return result
end
make_command_line(ary) click to toggle source

Generate command line string with special characters escaped.


Arguments:

  • (required) ary: Array containing String objects

Returns

String object

# File lib/bio/command.rb, line 134
def make_command_line(ary)
  if windows_platform? then
    make_command_line_windows(ary)
  else
    make_command_line_unix(ary)
  end
end
make_command_line_unix(ary) click to toggle source

Generate command line string with special characters escaped for UNIX shells.


Arguments:

  • (required) ary: Array containing String objects

Returns

String object

# File lib/bio/command.rb, line 158
def make_command_line_unix(ary)
  ary.collect { |str| escape_shell_unix(str) }.join(" ")
end
make_command_line_windows(ary) click to toggle source

Generate command line string with special characters escaped for cmd.exe on Windows.


Arguments:

  • (required) ary: Array containing String objects

Returns

String object

# File lib/bio/command.rb, line 148
def make_command_line_windows(ary)
  ary.collect { |str| escape_shell_windows(str) }.join(" ")
end
mktmpdir(prefix_suffix = nil, tmpdir = nil) { |path| ... } click to toggle source

Backport of Dir.mktmpdir in Ruby 1.9.

Same as Dir.mktmpdir(prefix_suffix) in Ruby 1.9.


Arguments:

  • (optional) prefix_suffix: String (or Array, etc.)

  • (optional) tmpdir: String: temporary directory's path

# File lib/bio/command.rb, line 573
def mktmpdir(prefix_suffix = nil, tmpdir = nil, &block)
  begin
    Dir.mktmpdir(prefix_suffix, tmpdir, &block)
  rescue NoMethodError
    # backported from Ruby 1.9.2-preview1.
    # ***** Below is excerpted from Ruby 1.9.2-preview1's lib/tmpdir.rb ****
    # ***** Be careful about copyright. ****
    case prefix_suffix
    when nil
      prefix = "d"
      suffix = ""
    when String
      prefix = prefix_suffix
      suffix = ""
    when Array
      prefix = prefix_suffix[0]
      suffix = prefix_suffix[1]
    else
      raise ArgumentError, "unexpected prefix_suffix: #{prefix_suffix.inspect}"
    end
    tmpdir ||= Dir.tmpdir
    t = Time.now.strftime("%Y%m%d")
    n = nil
    begin
      path = "#{tmpdir}/#{prefix}#{t}-#{$$}-#{rand(0x100000000).to_s(36)}"
      path << "-#{n}" if n
      path << suffix
      Dir.mkdir(path, 0700)
    rescue Errno::EEXIST
      n ||= 0
      n += 1
      retry
    end

    if block_given?
      begin
        yield path
      ensure
        remove_entry_secure path
      end
    else
      path
    end
    # ***** Above is excerpted from Ruby 1.9.2-preview1's lib/tmpdir.rb ****
  end
end
new_http(address, port = 80) click to toggle source

Same as:

Net::HTTP.new(address, port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.


Arguments:

  • (required) address: String containing host name or IP address

  • (optional) port: port (sanme as Net::HTTP::start)

Returns

(same as Net::HTTP.new except for proxy support)

# File lib/bio/command.rb, line 782
def new_http(address, port = 80)
  uri = URI.parse("http://#{address}:#{port}")
  # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
  # If the spec of open-uri.rb would be changed, we should change below.
  if proxyuri = uri.find_proxy then
    raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
    Net::HTTP.new(address, port, proxyuri.host, proxyuri.port)
  else
    Net::HTTP.new(address, port)
  end
end
no_fork?() click to toggle source

CAUTION Bio::Command INTERNAL USE ONLY. Users must NOT use the method. The method will be removed when it is not needed.

Checks if the OS does not support fork(2) system call. When not supported, it returns true. When supported or unknown, it returns false or nil.

Known issues:

  • It might make a mistake in minor platforms/architectures/interpreters.


Returns

true, false or nil.

# File lib/bio/command.rb, line 80
def no_fork?
  if (defined?(@@no_fork) && @@no_fork) or
      windows_platform? or /java/i =~ RUBY_PLATFORM then
    true
  else
    false
  end
end
post(uri, data, header = {}) click to toggle source

Same as: Net::HTTP.post(uri, params) and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Default Content-Type is application/octet-stream. Content-Length is automatically set by default.) uri must be a URI object, data must be a String, and header must be a hash.


Arguments:

  • (required) uri: URI object or String

  • (optional) data: String containing data

  • (optional) header: Hash containing header strings

Returns

(same as Net::HTTP::post)

# File lib/bio/command.rb, line 967
def post(uri, data, header = {})
  unless uri.is_a?(URI)
    uri = URI.parse(uri)
  end

  hash = {
    'Content-Type'   => 'application/octet-stream',
    'Content-Length' => data.length.to_s
  }
  hash.update(header)

  start_http_uri(uri) do |http|
    http.post(uri.path, data, hash)
  end
end
post_form(uri, params = nil, header = {}) click to toggle source

Same as: Net::HTTP.post_form(uri, params) and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. In addition, header can be set. (Note that Content-Type and Content-Length are automatically set by default.) uri must be a URI object, params must be a hash, and header must be a hash.


Arguments:

  • (required) uri: URI object or String

  • (optional) params: Hash containing parameters

  • (optional) header: Hash containing header strings

Returns

(same as Net::HTTP::post_form)

# File lib/bio/command.rb, line 841
def post_form(uri, params = nil, header = {})
  unless uri.is_a?(URI)
    uri = URI.parse(uri)
  end

  data = make_cgi_params(params)

  hash = {
    'Content-Type'   => 'application/x-www-form-urlencoded',
    'Content-Length' => data.length.to_s
  }
  hash.update(header)

  start_http_uri(uri) do |http|
    http.post(uri.path, data, hash)
  end
end
query_command(cmd, query = nil, options = {}) click to toggle source

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

Automatically select popen for Ruby 1.9 or Windows environment and fork for the others.

Available options:

:chdir => "path" : changes working directory to the specified path.

Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

  • (optional) options: Hash

Returns

String or nil

# File lib/bio/command.rb, line 442
def query_command(cmd, query = nil, options = {})
  if RUBY_VERSION >= "1.9.0" then
    return query_command_popen(cmd, query, options)
  elsif no_fork? then
    query_command_popen(cmd, query, options)
  else
    begin
      query_command_fork(cmd, query, options)
    rescue NotImplementedError
      # fork(2) not implemented
      @@no_fork = true
      query_command_fork(cmd, query, options)
    end
  end
end
query_command_fork(cmd, query = nil, options = {}) click to toggle source

This method is internally called from the #query_command method. In normal case, use #query_command, and do not call this method directly.

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

Fork (by using IO.popen(“-”)) and exec is used to execute the program.

See the document of #query_command for available options.

See the document of #call_command_fork for the security and Ruby version specific issues.


Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

  • (optional) options: Hash

Returns

String or nil

# File lib/bio/command.rb, line 507
def query_command_fork(cmd, query = nil, options = {})
  ret = nil
  call_command_fork(cmd, options) do |io|
    io.sync = true
    io.print query if query
    io.close_write
    ret = io.read
  end
  ret
end
query_command_open3(cmd, query = nil) click to toggle source

Executes the program via Open3.popen3 with the query (String) given to the stain, waits the program termination, and returns the data from stdout and stderr as an array of the strings.

You would use this method only when you really need to get stderr.


Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

Returns

Array containing 2 objects: output string (or nil) and stderr string (or nil)

# File lib/bio/command.rb, line 529
def query_command_open3(cmd, query = nil)
  errorlog = nil
  cmd = safe_command_line_array(cmd)
  Open3.popen3(*cmd) do |pin, pout, perr|
    perr.sync = true
    t = Thread.start { errorlog = perr.read }
    begin
      pin.print query if query
      pin.close
      output = pout.read
    ensure
      t.join
    end
    [ output, errorlog ]
  end
end
query_command_popen(cmd, query = nil, options = {}) click to toggle source

This method is internally called from the #query_command method. In normal case, use #query_command, and do not call this method directly.

Executes the program with the query (String) given to the standard input, waits the program termination, and returns the output data printed to the standard output as a string.

See the document of #query_command for available options.

See the document of #call_command_popen for the security and Ruby version specific issues.


Arguments:

  • (required) cmd: Array containing String objects

  • (optional) query: String

  • (optional) options: Hash

Returns

String or nil

# File lib/bio/command.rb, line 476
def query_command_popen(cmd, query = nil, options = {})
  ret = nil
  call_command_popen(cmd, options) do |io|
    io.sync = true
    io.print query if query
    io.close_write
    ret = io.read
  end
  ret
end
read_uri(uri) click to toggle source

Same as OpenURI.open_uri(uri).read and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.


Arguments:

  • (required) uri: URI object or String

Returns

String

# File lib/bio/command.rb, line 706
def read_uri(uri)
  OpenURI.open_uri(uri).read
end
remove_entry_secure(path, force = false) click to toggle source

Same as FileUtils.remove_entry_secure after Ruby 1.8.3. In Ruby 1.8.2 or previous version, it only shows warning message and does nothing.

It is strongly recommended using Ruby 1.8.5 or later.


Arguments:

  • (required) path: String

  • (optional) force: boolean

# File lib/bio/command.rb, line 555
def remove_entry_secure(path, force = false)
  begin
    FileUtils.remove_entry_secure(path, force)
  rescue NoMethodError
    warn "The temporary file or directory is not removed because of the lack of FileUtils.remove_entry_secure. Use Ruby 1.8.3 or later (1.8.5 or later is strongly recommended): #{path}"
    nil
  end
end
safe_command_line_array(ary) click to toggle source

Returns an Array of command-line command and arguments that can be safely passed to Kernel.exec etc. If the given array is already safe (or empty), returns the given array.


Arguments:

  • (required) ary: Array

Returns

Array

# File lib/bio/command.rb, line 169
def safe_command_line_array(ary)
  ary = ary.to_ary
  return ary if ary.size >= 2 or ary.empty?
  if ary.size != 1 then
    raise 'Bug: assersion of ary.size == 1 failed'
  end
  arg0 = ary[0]
  begin
    arg0 = arg0.to_ary
  rescue NoMethodError
    arg0 = [ arg0, arg0 ]
  end
  [ arg0 ]
end
start_http(address, port = 80, &block) click to toggle source

Same as:

Net::HTTP.start(address, port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set.


Arguments:

  • (required) address: String containing host name or IP address

  • (optional) port: port (sanme as Net::HTTP::start)

Returns

(same as Net::HTTP::start except for proxy support)

# File lib/bio/command.rb, line 758
def start_http(address, port = 80, &block)
  uri = URI.parse("http://#{address}:#{port}")
  # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
  # If the spec of open-uri.rb would be changed, we should change below.
  if proxyuri = uri.find_proxy then
    raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
    http = Net::HTTP.Proxy(proxyuri.host, proxyuri.port)
  else
    http = Net::HTTP
  end
  http.start(address, port, &block)
end
start_http_uri(uri, &block) click to toggle source

Same as:

Net::HTTP.start(uri.address, uri.port)

and it uses proxy if an environment variable (same as OpenURI.open_uri) is set. It supports https.

Note: This method ignores uri.path. It only uses uri.address and uri.port.


Arguments:

  • (required) uri: URI object or String containing URI

Returns

(same as Net::HTTP::start except for proxy and https support)

# File lib/bio/command.rb, line 724
def start_http_uri(uri, &block)
  unless uri.is_a?(URI)
    uri = URI.parse(uri)
  end

  # Note: URI#find_proxy is an unofficial method defined in open-uri.rb.
  # If the spec of open-uri.rb would be changed, we should change below.
  if proxyuri = uri.find_proxy then
    raise 'Non-HTTP proxy' if proxyuri.class != URI::HTTP
    klass = Net::HTTP.Proxy(proxyuri.host, proxyuri.port)
  else
    klass = Net::HTTP
  end

  http = klass.new(uri.host, uri.port)
  case uri.scheme
  when 'https'
    http.use_ssl = true
  end

  http.start(&block)
end
windows_platform?() click to toggle source

CAUTION Bio::Command INTERNAL USE ONLY. Users must NOT use the method. The method will be removed when it is not needed.

Checks if the program is running on Microsoft Windows. If Windows, returns true. Otherwise, returns false. Note that Cygwin is not treated as Windows.

Known issues:

  • It might make a mistake in minor platforms/architectures/interpreters.

  • When running JRuby on Cygwin, the result is unknown.


Returns

true or false

# File lib/bio/command.rb, line 50
def windows_platform?
  case RUBY_PLATFORM
  when /(?:mswin|bccwin|mingw)(?:32|64)/i
    true
  when /java/i
    # Reference: Redmine's platform.rb
    # http://www.redmine.org/projects/redmine/repository/revisions/1753/entry/trunk/lib/redmine/platform.rb
    if /windows/i =~ (ENV['OS'] || ENV['os']).to_s then
      true
    else
      false
    end
  else
    false
  end
end