[Shirley Kidman]

Has anyone had any experience with the jerichoHTML parser?
Http://jerichohtml.sourceforge.net/doc/index.html
Im trying to make my java prog access a web page that is behind a proxy at work
I have vpn'd in fine
But im wondering where i can state my un and pw for the proxy
I tried String sourceUrlString="http://user:pass@sample.com.au/index.php";
But still seem to be getting Exception in thread "main" java.io.IOException: Server returned HTTP response code: 401 for URL:
Any clues?

[Nicolas Murphy]

Shirley Kidman, you have to set the right headers instead of modifying the url
do you have ethereal or any other network sniffer available?
then install it.

[Shirley Kidman]

Ill have a look for a osx one

[Nicolas Murphy]

Shirley Kidman, sniff the traffic from your web browser see what header it sets, sniff the trafic from your java program and see what it sets, fill in the missing ones

[Shirley Kidman]

What am i trying to look for on the network?

[Nicolas Murphy]

Shirley Kidman, most probably a Proxy-Authentication header...
unless your proxy uses NTLM and is microsoft only

[Shirley Kidman]

No im on osx

[Nicolas Murphy]

Shirley Kidman, note however that you get a 401 and not a 407 so I am not sure what your proxy expects

[Shirley Kidman]

And im vpn in at the moment

[Nicolas Murphy]

Shirley Kidman, ok, NTML works with IE (and that used to be available on mac)
still, install a good network sniffer and find out

[Shirley Kidman]

Im sorry im a little lost when u refer to header, what part of my java program is the header?
Ok ill grab that one
Im just wondering where im specifying that info in my java prog at the moment
Or how i specify it.

[Nicolas Murphy]

Shirley Kidman, if you are using URLConnection you have to set headers on it...

[Shirley Kidman]

Ok, it might have a default setting in the libraries somewhere already that was preset

[Nicolas Murphy]

Shirley Kidman, URLConnection.setRequestProperty can be useful...

[Shirley Kidman]

But basically ill have to make that header, match the one that i receive when i browse to the page normally?
Ok
Thanks heaps Nicolas Murphy

[Nicolas Murphy]

Shirley Kidman, and please note that proxy-authentication headers of type Basic are base64 encoded
so they are just trivially scrambled clear text and trivial to get

[Shirley Kidman]

Trivial as in impossible?

[Nicolas Murphy]

Shirley Kidman, trivial as in very easy

[Shirley Kidman]

Ah

[Nicolas Murphy]

you take 3 bytes and create 4 bytes, just some bit shifting
just to make sure that the string is 7 bit clean

[Shirley Kidman]

Will this header file be different if logged on from different locations?

[Nicolas Murphy]

probably not, depends on what header it is...
install a network sniffer so that you can find out

[Shirley Kidman]

The programs is going to basically log into an online roster page, strip the html tags out and re format the text, eventually into a format that will import into ical