Post by Kevin CarlsonNever mind. I don't think the ClientCookie is going to help me
here. As someone had eluded earlier in this thread, the cookie is not
coming back as a header. I was looking through the headers and
couldn't see a Set-Cookie header -- thought I was going crazy.
As it turns out, one of the pages in the frameset contains Javascript
that creates the cookie. I can parse that for the Cookie and send it
back in subsequent requests.
[...]
Yes, that's a pain. I'm contemplating writing a Javascript/Python
thing that would solve some of these problems for HTTP cookies and
HTML forms. Vapourware.
Anyway, there is still some point in ClientCookie even for people who
have Javascript cookies, though. ATM, it knows which cookies to
accept and return to the server, and can do that for all requests --
so you just have to set a cookie once, then forget about it and just
call urlopen (or OpenerDirector.open). In future, I imagine it will
give you access to a proper constructor for Cookie structs.
import urllib2, urlparse
import ClientCookie # 0.4.1a required
cs = ClientCookie.Cookies()
opener = ClientCookie.build_opener(ClientCookie.HTTPHandler(c))
request = urllib2.Request("http://www.example.com")
response = opener.open(request)
cookies = your_cookie_parsing_routine(request, response)
for cookie in cookies:
cs.set_cookie_if_ok(cookie, request)
# go on using opener.open() -- cookies get sent back when they should be
# (and any that arrive via the traditional HTTP route will be set, if
# appropriate)
opener.open("http://www.example.com/blah.html")
opener.open("http://www.example.com/foo.html")
def your_cookie_parsing_routine(request, response):
data = response.read()
# Parse name, value, domain, path from data (ie. the HTML) here.
# If domain and path are not specified, set the domain_specified and
# path_specified flags accordingly, and use the request values:
req_domain, req_path = urlparse.urlparse(request,get_full_url())
if not domain_specified:
domain = req_domain
if not path_specified:
path = req_path
# Create a Cookie struct:
# (ClientCookie will aquire a better way of doing this eventually)
c = Cookie(0, name, value,
None, 0,
domain, domain_specified, domain.startswith("."),
path, path_specified,
secure, # true if must only be sent back via https
time.time()+(3600*24*365), # expires
0, "", "", {})
return [c] # you may be parsing out more than one, of course
Hmm, probably ClientCookie's cookie parsers and constructors should be
factored out of Cookies, so you can supply your own parser and not
need to construct Cookie structs by hand... more refactoring ahead
(but no more interface changes, I hope!).
John