Monday, January 02, 2006

Download Oracle XE with wget

I needed to download Oracle XE using wget mainly because of the limit rate option, but also because i can run it in the background and files can be resumed at any time. The limit rate option allows me to not use almost all the available Internet bandwidth, so i tried to download it but i noticed that you can't get the file directly from http://download.oracle.com/otn/nt/oracle10g/xe/OracleXE.exe (this link is for windows platforms) without beeing authenticated, you must have a cookie if you already signed in, otherwise you will be forwarded to
https://profile.oracle.com/jsp/realms/otnLogin.jsp, where there is a form asking for username and password that post to https://profile.oracle.com/jsp/reg/loginHandler.jsp sending this and other additional variables for authentication and finally redirection to the url that we came from (as you can se below in the redirectUrl variable) that in this case was the file to download.

If we look at the talk between browser and server when sending the form we can see the POST variables :


https://profile.oracle.com/jsp/reg/loginHandler.jsp

POST /jsp/reg/loginHandler.jsp HTTP/1.1
Host: profile.oracle.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.7.12) Gecko/20050915 Firefox/1.0.7
Accept: text/xml,application/xml,application/xhtml+xml,
text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: https://profile.oracle.com/jsp/realms/otnLogin.jsp?
remoteIp=63.XXX.XX.XX&globalId=&redirectUrl=http%3a%2f%2f
download-east.oracle.com%3a80%2fotn%2fnt%2foracle10g%2fxe%2f
OracleXE.exeCookie: JSESSIONID=8d920d0fce90b20ff91eaec4cc2aa
dae09395793893.mkPOqNeKaMTJqA5xp6iImQXHcxaNa3iUfiP-j45e8Ojbg
Pf1i4P9f34ImkPOqNeKaMTJqA5xp6iImQXH8N4MbhyKax0Lc34xf2bdeNf+l
OXgnkrDqRfMmlfDpQSxn6jAmljGr5XDqQLvpAe_;s_cc=true; s_sq=orac
leotnlive%2Coracleglobal%3D%2526pid%253Dhttp%25253A//www.ora
cle.com/technology/software/htdocs/xe_lic.html%25253F/techno
logy/software/products/database/xe/htdocs/102xewinsoft.html
%2526oid%253Dhttp%25253A//www.oracle.com/technology/software
/products/database/xe/htdocs/102xewinsoft.html%2526ot%253DA
Content-Type: application/x-www-form-urlencoded
Content-Length: 324

remoteIp=63.XXX.XX.XX&globalId=&redirectUrl=http%3A%2F%2Fdo
wnload-east.oracle.com%2Fotn%2Fnt%2Foracle10g%2Fxe%2FOracle
XE.exe&onFailRedirectUrl=%2Fjsp%2Frealms%2FotnWrongPassword
.jsp%3Fnexturl%3Dhttp%3A%2F%2Fdownload-east.oracle.com%2Fot
n%2Fnt%2Foracle10g%2Fxe%2FOracleXE.exe&username=camafla&pas
sword=xxxxxx&submit=Continue



with that POST variables and URL i compose the following
wget command:


wget -v -c --limit-rate=3K --no-check-certificate --post-data="remoteIp=63.XXX.XX.XX&globalId=&redirectUrl=http://download.oracle.com/otn/nt/oracle10g/xe/10201/OracleXE.exe&onFailRedirectUrl=%2Fjsp%2Frealms%2FotnWrongPassword.jsp%3Fnexturl%3Dhttp%3A%2F%2Foracleheva1.oracle.com%2Fotn%2Fnt%2Foracle10g%2Fxe%2FOracleXE.exe&username=camafla&password=xxxxxx&submit=Continue" https://profile.oracle.com/jsp/reg/loginHandler.jsp -o oraxe.log &


-c , is used when you are resuming the download
--limit-rate=3K, limit to 3 Kbytes/sec
--no-check-certificate, it doesnt work verifying the server
certificate
-o oraxe.log, for output
& to run on background

At the end of the POST variables you can find "...username=camafla&password=xxxxx..." , you have to change
that with your personal information (also remoteIp=63.XXX.XX.XX).

The redirectUrl=http://download.oracle.com/otn/nt/oracle10g/xe/10201/OracleXE.exe variable may change but you just have to check with a browser whats the actual url.

13 comments:

Laurent Schneider said...

> If we look at the talk between browser and server

How?

Carlos said...

I use the
live HTTP headers extension
, it lets you watch the request/response messages between browser and server.

Laurent Schneider said...

ok, thanks for the hint, even if it is not as convenient as wget --load-cookies, it is still a good thing to know, just in case load-cookies will stop working one day...

Anonymous said...

Is there a plugin for konqueror?

Carlos said...

I'm sorry, I don't think so. At least not live http headers extension. Why would you use konqueror anyway?.

Anonymous said...

Man, really thanks a lot!

Anonymous said...

There is another way to do this that I found. You might find it easier or not. I was forced to use the command line to download XE and was at a quandry of how to do it. I tried lynx 3 times and was not getting anywhere. I finally used linx to get past the login screen and start the download process, then I opened a new ssh window in the same session and used wget to pull the rpm directly. Since I was already authenticated, it downloaded the file directly.

For me it was faster than forming a key value url string.

Anonymous said...

Thank you, I had the same problem and your solution was very useful.

Anonymous said...
This comment has been removed by a blog administrator.
Unknown said...
This comment has been removed by the author.
Unknown said...

some reason the url in the message I was trying to post didn't show up properly - ignore the previous post.

Imre said...

get parameters work as good:

just replace XXX-s and file url

wget.exe -c --no-check-certificate "https://profile.oracle.com/jsp/reg/loginHandler.jsp?remoteIp=90.XXX.XXX.XXX&globalId=&redirectUrl=http://download-llnw.oracle.com:80/otn/nt/oracle11g/win32_11gR1_database.zip&username=XXX&password=XXX"

Patschius said...

many thx!