Generate cookie file from Watir



Disclaimer: this is the extended post from a solution I gave on StackOverflow

So I had to build a scraper which has to download a file.

It is pretty easy to download a file from a browser, set the download directory, navigate the page, if the file is after a link, just click it. But there is a catch, you can ask the browser to click the link and the download will follow, but how can you tell when its done, and if there are several files to download, which one is the latest?

Most of what I've seen is something like

file_list = list_files_in_download_dir
diff = []
10.times do
file_list_now = list_files_in_download_dir
diff = file_list_now - file_list
break if diff.length.positive?

puts "The files are: #{diff}"

But having the file when it has been downloaded is top priority to me, maybe also to you.

So I guess another approach must be taken. I like to navigate using a browser, so maybe I could make the browser do must of the grunt work and when I'm ready I can download the file myself! But now I need to have the browser's cookie to identify myself as the proper user. So I had to take the cookies from Watir and pass them to Typhoeus

Since I use Typhoeus and Typhoeus wraps curl, from another article I learned that curl uses Mozilla style cookie files

So, if your Watir instance is browser and the file to which you are going to write to is file you can do

browser.cookies.to_a.each do |ch| 
terms = []
terms << ch[:domain]
terms << ch[:same_site].nil? ? 'FALSE' : 'TRUE'
terms << ch[:path]
terms << ch[:secure] ? 'TRUE' : 'FALSE'
terms << ch[:expires].to_i.to_s
terms << ch[:name]
terms << ch[:value]
file.puts terms.join("\t")