Ruby/curl link expander method is downloading the full target url -
i made handy little link expander using curl within ruby (sintra) app.
def curbexpand(link) result = curl::easy.new(link) begin result.headers["user-agent"] = "..." result.verbose = true result.follow_location = true result.max_redirects = 3 result.connect_timeout = 5 result.perform return result.last_effective_url # returns final destination url after x redirects... rescue return link puts "xxxxxxxxxxxxxxxxxxx error parsing link xxxxxxxxxxxxxxxxxxxxxxxxxxx" end end
the problem have geniuses using url shorteners link .exe's , .dmg's fine looks curl script above waiting full response returned (i.e. 1gb file!) before returning url. don't want use third party link expander api's have significant volume of links expand.
anyone know how can tweak curb find url rather waiting full response?
i've done want using using net::http
process "head" requests, , redirects way. advantage head not return content, headers.
from the docs:
head(path, initheader = nil) gets header path on connected-to host. header hash { ‘accept’ => ‘/’, … }. method returns net::httpresponse object. method never raises exception. response = nil net::http.start('some.www.server', 80) {|http| response = http.head('/index.html') } p response['content-type']
combine example in net::http docs following redirection, , should able find landing url.
you can use curl::http_head
accomplish same thing.
Comments
Post a Comment