Ruby Net :: HTTP time out

Question

Estoy tratando de escribir mi primer programa Ruby, pero tengo un problema. El código tiene que descargar 32 archivos MP3 a través de HTTP. De hecho, descarga unos pocos, luego se agota el tiempo de espera.

Intenté establecer un período de tiempo de espera, pero no hace la diferencia. Al ejecutar el código en Windows, Cygwin y Mac OS X tienen el mismo resultado.

Este es el código:

require 'rubygems' 
require 'open-uri' 
require 'nokogiri' 
require 'set' 
require 'net/http' 
require 'uri' 

puts "\n Up and running!\n\n" 

links_set = {} 

pages = ['http://www.vimeo.com/siai/videos/sort:oldest', 
    'http://www.vimeo.com/siai/videos/page:2/sort:oldest', 
    'http://www.vimeo.com/siai/videos/page:3/sort:oldest'] 

pages.each do |page| 
    doc = Nokogiri::HTML(open(page)) 
    doc.search('//*[@href]').each do |m| 
    video_id = m[:href] 
    if video_id.match(/^\/(\d+)$/i) 
    links_set[video_id[/\d+/]] = m.children[0].to_s.split(" at ")[0].split(" -- ")[0] 
    end 
    end 
end 

links = links_set.to_a 

p links 

cookie = '' 
file_name = '' 

open("http://www.tubeminator.com") {|f| 
    cookie = f.meta['set-cookie'].split(';')[0] 
} 

links.each do |link| 
    open("http://www.tubeminator.com/ajax.php?function=downloadvideo&url=http%3A%2F%2Fwww.vimeo.com%2F" + link[0], 
    "Cookie" => cookie) {|f| 
     puts f.read 
    } 

    open("http://www.tubeminator.com/ajax.php?function=convertvideo&start=0&duration=1120&size=0&format=mp3&vq=high&aq=high", 
    "Cookie" => cookie) {|f| 
     file_name = f.read 
    } 
    puts file_name 

    Net::HTTP.start("www.tubeminator.com") { |http| 
    #http.read_timeout = 3600 # 1 hour 
    resp = http.get("/download-video-" + file_name) 
    open(link[1] + ".mp3", "wb") { |file| 
     file.write(resp.body) 
    } 
    } 
end 

puts "\n Yay!!" 

Y esta es la excepción:

/Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:140:in `rescue in rbuf_fill': Timeout::Error (Timeout::Error) 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:134:in `rbuf_fill' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:116:in `readuntil' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/protocol.rb:126:in `readline' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:2138:in `read_status_line' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:2127:in `read_new' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:1120:in `transport_request' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:1106:in `request' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:312:in `block in open_http' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/net/http.rb:564:in `start' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:306:in `open_http' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:767:in `buffer_open' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:201:in `catch' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:669:in `open' 
from /Users/test/.rvm/rubies/ruby-1.9.2-preview1/lib/ruby/1.9.1/open-uri.rb:33:in `open' 
from test.rb:38:in `block in <main>' 
from test.rb:37:in `each' 
from test.rb:37:in `<main>' 

También apreciaría sus comentarios en el resto del código.

Answer 1

Su tiempo de espera no está en el código para el que estableció el tiempo de espera. Es aquí, donde se utiliza open-uri:

open("http://www.tubeminator.com/ajax.php?function=downloadvideo&url=http%3A%2F%2Fwww.vimeo.com%2F" + link[0], 

se puede establecer un tiempo de espera de lectura de Open-URI de este modo:

#!/usr/bin/ruby1.9 

require 'open-uri' 

open('http://stackoverflow.com', 'r', :read_timeout=>0.01) do |http| 
    http.read 
end 

# => /usr/lib/ruby/1.9.0/net/protocol.rb:135:in `sysread': \ 
# => execution expired (Timeout::Error) 
# => ... 
# =>   from /tmp/foo.rb:5:in `<main>' 

:read_timeout es nuevo para Ruby 1.9 (que no es en Ruby 1.8) . 0 o nil significa "sin tiempo de espera".

Answer 2

Para Ruby 1.8 Lo usé para resolver mis problemas de tiempo de espera. La extensión de la clase Net :: HTTP en mi código y la reinicialización con parámetros predeterminados, incluida una inicialización de mi propio read_timeout, debería mantener las cosas en su sano juicio, creo.

require 'net/http' 

# Lengthen timeout in Net::HTTP 
module Net 
    class HTTP 
     alias old_initialize initialize 

     def initialize(*args) 
      old_initialize(*args) 
      @read_timeout = 5*60  # 5 minutes 
     end 
    end 
end