Must we include urllib just to decode a URL-encoded string, when using Requests?

[Dotan Cohen]

I am using the Requests module to access remote URLs. Sometimes I need
to URL-decode or URL-encode a string (via RFC 3986). Must I import
urllib or urllib2 just to use their quote() and unquote() methods?
Does not Requests have such an ability, and perhaps I just cannot find
it?

On Stack Overflow [1] I found this wonderful function:
def unquote(url):
return re.compile('%([0-9a-fA-F]{2})',re.M).sub(lambda m:
chr(int(m.group(1),16)), url)

I am already importing the 're' module so that is not an issue. I am
concerned, though, that this might not work for some non-ASCII
characters such as some esoteric symbols or Korean/Chinese/Japanese
characters.

[1] http://stackoverflow.com/a/15627281/343302

--
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com

[Robert Kern]

Post by Dotan Cohen
I am using the Requests module to access remote URLs. Sometimes I need
to URL-decode or URL-encode a string (via RFC 3986). Must I import
urllib or urllib2 just to use their quote() and unquote() methods?

Yes. Do you think there is a problem with doing so?

[Dotan Cohen]

Post by Robert Kern
Yes. Do you think there is a problem with doing so?

I'm pretty sure that Requests will use either urllib or urllib2,
depending on what is available on the server. I would like to use
whatever Requests is currently using, rather than import the other.
Can I tell which library Requests is currently using and use that?

--
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com

[Robert Kern]

Post by Dotan Cohen

Post by Robert Kern
Yes. Do you think there is a problem with doing so?

I'm pretty sure that Requests will use either urllib or urllib2,
depending on what is available on the server.

No, it doesn't. It gets its quote() function from urllib always.

Post by Dotan Cohen
I would like to use
whatever Requests is currently using, rather than import the other.
Can I tell which library Requests is currently using and use that?

The only thing I can think that you are talking about is the difference between
Python 2 and Python 3. In Python 2, it's urllib.quote() and in Python 3, it's
urllib.parse.quote(), but that's a Python-version issue, not something to do
with requests, per se. requests does have a compatibility layer, internally,
that pastes over those issues, but I don't think that is intended to be a stable
public API that you should rely on. You should handle that kind of switch
yourself if you care about compatibility across both versions of Python.

https://github.com/kennethreitz/requests/blob/master/requests/compat.py#L86

[Dotan Cohen]

Post by Robert Kern

Post by Dotan Cohen
I'm pretty sure that Requests will use either urllib or urllib2,
depending on what is available on the server.

No, it doesn't. It gets its quote() function from urllib always.

I see, thanks. Then that is what I will do as well!

Post by Robert Kern

Post by Dotan Cohen
I would like to use
whatever Requests is currently using, rather than import the other.
Can I tell which library Requests is currently using and use that?

The only thing I can think that you are talking about is the difference
between Python 2 and Python 3. In Python 2, it's urllib.quote() and in
Python 3, it's urllib.parse.quote(), but that's a Python-version issue, not
something to do with requests, per se. requests does have a compatibility
layer, internally, that pastes over those issues, but I don't think that is
intended to be a stable public API that you should rely on. You should
handle that kind of switch yourself if you care about compatibility across
both versions of Python.
https://github.com/kennethreitz/requests/blob/master/requests/compat.py#L86

Great, thank you Robert. I see that Requests is calling both urllib
and urllib2. For some reason I thought that is rather wasteful and
should be avoided. I was probably wrong!

--
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com

[Dotan Cohen]

On Thu, Jun 13, 2013 at 4:44 PM, Burak Arslan

paste this to your python console, it'll show you what modules requests
import sys
p = set(sys.modules)
import requests
print(m)

Thank you. Python is a beautiful language, I cannot believe that the
set(sys.modules)-p line does what it does!

Interestingly, on my system with Python3 neither urllib nor urllib2
are imported, only urllib3 which I had not heard of until now:

__future__
_json
atexit
cgi
chardet
html
http.cookiejar
http.cookies
json
json.decoder
json.encoder
json.scanner
logging
mimetypes
netrc
queue
requests
requests.api
requests.auth
requests.compat
requests.cookies
requests.defaults
requests.exceptions
requests.hooks
requests.models
requests.sessions
requests.status_codes
requests.structures
requests.utils
shlex
six
six.moves
threading
urllib3
urllib3._collections
urllib3.connectionpool
urllib3.exceptions
urllib3.filepost
urllib3.packages
urllib3.packages.mimetools_choose_boundary
urllib3.packages.ssl_match_hostname
urllib3.poolmanager
urllib3.request
urllib3.response
urllib3.util


--
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com

[Burak Arslan]

Post by Dotan Cohen

Post by Robert Kern
Yes. Do you think there is a problem with doing so?

I'm pretty sure that Requests will use either urllib or urllib2,
depending on what is available on the server. I would like to use
whatever Requests is currently using, rather than import the other.
Can I tell which library Requests is currently using and use that?

paste this to your python console, it'll show you what modules requests
imports:


import sys
p = set(sys.modules)
import requests
for m in sorted(set(sys.modules) - p):
print(m)


burak