select.select question

[Heiko Wundram]

select can see only data that's readable from the system level
device (the socket, that is.) Your data is in a file object, C
library buffer in your process' space. That's my guess.
Python's select function cleverly supports rfile as an input, but
it isn't so clever as to actually be able to tell you if rfile is
readable; rather, it can only tell you if rfile's device is readable.

Yep... I tried that out just now, and it works, when I put the rfile on
unbuffered. I had to switch to the lower level functions (recv) anyway,
as the file-object functions don't return a length in most (????)
cases... Sometimes they do, and it works, but most of the time it
doesn't work... Strange.

Well... Using recv and send directly is no backbreaking experience, as I
can then quite easily switch to using an ssl socket some time later.

Thanks!

Heiko Wundram

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
URL: <http://mail.python.org/pipermail/python-list/attachments/20020823/3f26573d/attachment.pgp>

[Timothy O'Malley]

hola.

Most often, you should NOT set the socket to
non-blocking when using select.select()

See

http://www.timo-tasi.org/python/timeoutsocket.py

for simple timeout mechanisms with sockets. Look
forward to Python 2.3, where these timeouts will be
built in.

timO

# Set nonblocking and initialize data.
self.request.setblocking(0)

You shouldn't be making the socket non-blocking. Select
will return immediately for a non-blocking socket,
whether there's data there or not.

import select
from socket import *
s = socket(AF_INET, SOCK_STREAM)
s.connect(('www.google.com', 80))
s.setblocking(0)
select.select([s],[],[],1.0)
pauses for one second before returning on both my Linux box and my WinXP
box.
-Dave
P.S. Thanks for making Pyrex!

[Greg Ewing]

In article <3D69C8F2.60507 at something.invalid>, Greg Ewing
| You shouldn't be making the socket non-blocking. Select
| will return immediately for a non-blocking socket,
| whether there's data there or not.

s.setblocking(0)
from select import select
select([s], [], [], 1)

(... one second passes ...)
([], [], [])

You're quite right. I'm not sure where I got that idea
from; I seem to remember having problems with this on a
BSD system some years ago. Maybe things worked differently
back then.

[Greg Ewing]

# Set nonblocking and initialize data.
self.request.setblocking(0)

You shouldn't be making the socket non-blocking. Select
will return immediately for a non-blocking socket,
whether there's data there or not.

[Erno Kuusela]

In article <3D69C8F2.60507 at something.invalid>, Greg Ewing
<see_reply_address at something.invalid> writes:

| Heiko Wundram wrote:
|| # Set nonblocking and initialize data.
|| self.request.setblocking(0)

| You shouldn't be making the socket non-blocking. Select
| will return immediately for a non-blocking socket,
| whether there's data there or not.

i don't think so:

Python 2.1.3 (#1, Apr 20 2002, 10:14:34)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more information.

from socket import *
s = socket(AF_INET, SOCK_DGRAM)
s.bind(('', 12345))
s.setblocking(0)
from select import select
select([s], [], [], 1)

(... one second passes ...)
([], [], [])

putting it in nonblocking mode will let you catch
programming mistakes where you accidentally do something
blocking by mistake.

also i remember reading somewhere that recv on a socket can infact
block after select has claimed it has incoming data waiting in some
weird conditions, so it is good practice to catch that case too. no
reference, sorry. (for example, select on a listening socket, returns
readable, RST arrives, accept blocks)

-- erno

[Heiko Wundram]

Hey to all!

I'm a little dumbfounded by the following problem I seem to be
encountering when using the select.select module. The code basically
looks like the following:

class RequestHandler(SocketServer.BaseRequestHandler):
def handle():

# Read in command and data length in blocking mode.
command, length = ...

# Set nonblocking and initialize data.
self.request.setblocking(0)
data = ""

# Read until data has been completely read
while len(data) < length:
(rlist,wlist,xlist) = select.select([self.rfile],[],[],5.0)

# Connection dumped.
if not rlist:
return

# Read piece of data.
data += self.rfile.read(length-len(data))

# Do something with data and command.
...

The funny thing is that the command and the length (which are read in
standard/blocking mode) are read correctly, but the select call always
returns an empty list for the read available list. It is just the same
if I specify the socket directly (self.connection/self.request).

Doing a print self.rfile.read(1) right after the select doesn't block
and returns the next character correctly (which is there, I am sure of
that!), even though the rlist is empty! And setting the timeout to zero
just blocks the process as a whole, although self.rfile _is_ readable...

Has anybody experienced this problem before? What am I doing wrong? I
remember writing similar code in Perl, also using select, and that
worked just fine... That was pretty much the last time I was using
select, as I've found pythons threading SocketServer much more
convenient than writing a select based server... :)

Thanks for any answers!

btw. I want to use the select call to dump connections which don't react
anymore, or are so slow that it is unnecessary to keep a connection open
to the server. The handler has its own thread (it is called from a
SocketServer/ThreadingMixIn derived class). Dunno if this is a source of
problem, using a select in a subthread. Doesn't state that in the
documentation anywhere, though.

Yours,

Heiko Wundram

[brueckd]

# Set nonblocking and initialize data.
self.request.setblocking(0)

You shouldn't be making the socket non-blocking. Select
will return immediately for a non-blocking socket,
whether there's data there or not.

I don't think this is true. This code:

import select
from socket import *
s = socket(AF_INET, SOCK_STREAM)
s.connect(('www.google.com', 80))
s.setblocking(0)
select.select([s],[],[],1.0)

pauses for one second before returning on both my Linux box and my WinXP
box.

-Dave

P.S. Thanks for making Pyrex!

[Donn Cave]

Quoth Heiko Wundram <hewu5001 at stud.uni-saarland.de>:

| I'm a little dumbfounded by the following problem I seem to be
| encountering when using the select.select module. The code basically
| looks like the following:
|
| class RequestHandler(SocketServer.BaseRequestHandler):
| def handle():
|
| # Read in command and data length in blocking mode.
| command, length = ...
|
| # Set nonblocking and initialize data.
| self.request.setblocking(0)
| data = ""
|
| # Read until data has been completely read
| while len(data) < length:
| (rlist,wlist,xlist) = select.select([self.rfile],[],[],5.0)
|
| # Connection dumped.
| if not rlist:
| return
|
| # Read piece of data.
| data += self.rfile.read(length-len(data))
|
| # Do something with data and command.
| ...
|
| The funny thing is that the command and the length (which are read in
| standard/blocking mode) are read correctly, but the select call always
| returns an empty list for the read available list. It is just the same
| if I specify the socket directly (self.connection/self.request).
|
| Doing a print self.rfile.read(1) right after the select doesn't block
| and returns the next character correctly (which is there, I am sure of
| that!), even though the rlist is empty! And setting the timeout to zero
| just blocks the process as a whole, although self.rfile _is_ readable...

select can see only data that's readable from the system level
device (the socket, that is.) Your data is in a file object, C
library buffer in your process' space. That's my guess.
Python's select function cleverly supports rfile as an input, but
it isn't so clever as to actually be able to tell you if rfile is
readable; rather, it can only tell you if rfile's device is readable.

If you don't need the file object buffering, then you might as
well use the socket directly, and call the socket recv() method.
If you never use rfile, it won't read any data into its buffer.

Donn Cave, donn at drizzle.com