socket.unbind or socket.unlisten? - socket.error: (48, 'Address already in use')

[Laszlo Nagy]

I have a program that uses socket.bind() and socket.listen() frequently.
After that program stops, it is not able to bind() again for a while:

File "/home/gandalf/Python/Lib/orb/accesspoints/srvtcp.py", line 27, in
__init__
self.serversocket.bind((self.listen_address,self.port))
File "<string>", line 1, in bind
socket.error: (48, 'Address already in use')


The problem with this, is that this server program SOMETIMES need to be
restarted very quickly. I tried to find the solution in the socket
module. But there is no "socket.unbind" or "socket.unlisten". How can I
tell the OS that I do not want to listen on that address anymore, so
other programs can bind on it immediatelly?

(Yes I know that I can use setsockopt to allow listening multiple
sockets on the same address, but this is NOT what I need...)

Thanks,

Laszlo

[Hendrik van Rooyen]

Post by Laszlo Nagy
I have a program that uses socket.bind() and socket.listen() frequently.

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

This does the trick for me.

- Hendrik

[Jean-Paul Calderone]

Post by Laszlo Nagy
I have a program that uses socket.bind() and socket.listen() frequently.
File "/home/gandalf/Python/Lib/orb/accesspoints/srvtcp.py", line 27, in
__init__
self.serversocket.bind((self.listen_address,self.port))
File "<string>", line 1, in bind
socket.error: (48, 'Address already in use')
The problem with this, is that this server program SOMETIMES need to be
restarted very quickly. I tried to find the solution in the socket module.
But there is no "socket.unbind" or "socket.unlisten". How can I tell the OS
that I do not want to listen on that address anymore, so other programs can
bind on it immediatelly?
(Yes I know that I can use setsockopt to allow listening multiple sockets on
the same address, but this is NOT what I need...)

Actually, SO_REUSEADDR is probably just what you want. Since I can't see
your code and I don't know under what situations it fails, I can only guess
at the problem, but my guess is that you have connections from the first run
of your app left in the TIME_WAIT state and they are preventing you from
binding to the address again in the second run of your app. Setting the
SO_REUSEADDR flag on POSIX fixes this problem (don't set it on Windows,
though).

Jean-Paul

[Hendrik van Rooyen]

"Jean-Paul Calderone" <ex....n at di..od.com> wrote:

8<------------------------------

....... Setting the
SO_REUSEADDR flag on POSIX fixes this problem (don't set it on Windows,
though).

Why not? I have been merrily setting it, and I have not noticed anything weird.
(yet)

- Hendrik

[Laszlo Nagy]

Post by Hendrik van Rooyen
8<------------------------------

....... Setting the
SO_REUSEADDR flag on POSIX fixes this problem (don't set it on Windows,
though).

Why not? I have been merrily setting it, and I have not noticed anything weird.
(yet)

Please see my original post. I specifically stated that I do not want to
use setsockopt and be able to listen on the same port from many
processes. I knew that I could use SO_REUSEADDR, but I'm heistating to
do so. I must guarantee that only one process listens on a given port at
the same time.

Maybe I could use some kind of locking, but it would be too difficult:

- mutexes are great but they are platform dependent and they are not in
the standard lib
- the listening processes do not see each other's home directory so file
locking cannot be used for this
- these processes will probably listen on many ports at the same time,
it is also a problem with mutexes/file locks (who wants 50 lock files to
be created?)


Best,

Laszlo

[Steve Holden]

Post by Laszlo Nagy

Post by Hendrik van Rooyen
8<------------------------------

....... Setting the
SO_REUSEADDR flag on POSIX fixes this problem (don't set it on Windows,
though).

Why not? I have been merrily setting it, and I have not noticed anything weird.
(yet)

Please see my original post. I specifically stated that I do not want to
use setsockopt and be able to listen on the same port from many
processes. I knew that I could use SO_REUSEADDR, but I'm heistating to
do so. I must guarantee that only one process listens on a given port at
the same time.

This appears to demonstrate a misunderstanding of the purpose of the
SO_REUSEADDR flag.

SO_REUSEADDR should be used to avoid the TIME_WAIT state when the
current listening process terminates. If you do not use it then there
will be a certain period during which no other process can listen on
that port, and it will be reusable only after the TIME_WAIT is over.
This can easily be seen by using the netstat utility.

See "Pitfall 3" in

http://www.ibm.com/developerworks/linux/library/l-sockpit/index.html

Post by Laszlo Nagy
- mutexes are great but they are platform dependent and they are not in
the standard lib
- the listening processes do not see each other's home directory so file
locking cannot be used for this
- these processes will probably listen on many ports at the same time,
it is also a problem with mutexes/file locks (who wants 50 lock files to
be created?)

Complete red herring: there is no way for multiple processes to listen
to the same port: this would lead to ambiguity in the protocol stack.

regards
Steve

[Steve Holden]

Post by Laszlo Nagy

Post by Hendrik van Rooyen
8<------------------------------

....... Setting the
SO_REUSEADDR flag on POSIX fixes this problem (don't set it on Windows,
though).

Why not? I have been merrily setting it, and I have not noticed anything weird.
(yet)

Please see my original post. I specifically stated that I do not want to
use setsockopt and be able to listen on the same port from many
processes. I knew that I could use SO_REUSEADDR, but I'm heistating to
do so. I must guarantee that only one process listens on a given port at
the same time.

My previous reply assumed you are running some UNIX-like operating
system. If you are on Windows then Jean-Paul's advice stands, as Windows
*does* allow several processes to listen on the same port and randomly
delivers incoming connections to one of the listening processes.

I believe this is because Microsoft failed to understand the original
meaning of SO_REUSEADDR for their early TCP implementations, and
persisted with this ghastly error in the name of backwards
compatibility, justifying it by suggesting that listener pools could be
created. Or some such nonsense. Perhaps someone with more insight into
the development process could comment. It seems to me it's completely
bizarre.

However, under Windows 2000 and later you should find there's an
SO_EXCLUSIVEADDRUSE flag which you can use to ensure a single listener -
see http://msdn.microsoft.com/en-us/library/ms740621(VS.85).aspx. No
need for separate locks.

regards
Steve

[Hendrik van Rooyen]

Post by Steve Holden
My previous reply assumed you are running some UNIX-like operating
system. If you are on Windows then Jean-Paul's advice stands, as Windows
*does* allow several processes to listen on the same port and randomly
delivers incoming connections to one of the listening processes.
I believe this is because Microsoft failed to understand the original
meaning of SO_REUSEADDR for their early TCP implementations, and
persisted with this ghastly error in the name of backwards
compatibility, justifying it by suggesting that listener pools could be
created. Or some such nonsense. Perhaps someone with more insight into
the development process could comment. It seems to me it's completely
bizarre.
However, under Windows 2000 and later you should find there's an
SO_EXCLUSIVEADDRUSE flag which you can use to ensure a single listener -
see http://msdn.microsoft.com/en-us/library/ms740621(VS.85).aspx. No
need for separate locks.

Thanks Steve - I am not the OP, I was just curious as to why
Jean-Paul was saying what he did - so the only reason I have
not been bitten on windoze yet must be either because I am
lucky, or because my server side stuff is linux and there is the
occasional windows client. So I don't really listen on windows.

"a pool of listeners" - just think of the fun one can have trying to
keep state between them - would be a marvellous job if someone
else is paying top dollar by the hour - I can just see the team growing
as the need for new specialists are discovered as one goes along.

:-)

- Hendrik

[Grant Edwards]

Post by Steve Holden
I believe this is because Microsoft failed to understand the
original meaning of ___________________, and persisted with
this ghastly error in the name of backwards compatibility,
justifying it by suggesting that _________________________.

Somebody should have cards printed up...

[Aahz]

In article <QaidnUXDgsrtWhjUnZ2dnUVZ_vzinZ2d at posted.usinternet>,

Post by Grant Edwards

Post by Steve Holden
I believe this is because Microsoft failed to understand the
original meaning of ___________________, and persisted with
this ghastly error in the name of backwards compatibility,
justifying it by suggesting that _________________________.

Somebody should have cards printed up...

+1 QOTW

[Mark Wooding]

Post by Laszlo Nagy
I have a program that uses socket.bind() and socket.listen()
frequently. After that program stops, it is not able to bind() again for a

This is the usual TIME-WAIT problem. The TCP protocol requires one end
of the connection (whichever actually started the close) to keep a
record of it for a while after it closes, in order to avoid confusion
caused by old packets.

Post by Laszlo Nagy
The problem with this, is that this server program SOMETIMES need to be
restarted very quickly.

The usual solution is to set the SO_REUSEADDR socket option before
binding. This is safe for listening sockets.

Here's an interactive session.

In [1]: import socket as S

In [2]: def make_server_socket():
...: sk = S.socket(S.AF_INET, S.SOCK_STREAM)
...: sk.bind(('', 12345))
...: sk.listen(5)
...: return sk
...:

In [3]: def serve_client(sk):
...: (nsk, addr) = sk.accept()
...: nsk.send('Hello.\n')
...: nsk.close()
...:

In [4]: sk = make_server_socket()

In [5]: serve_client(sk)

(At this point, I connect to the server in another terminal.)

In [6]: sk.close()

In [7]: sk = make_server_socket()
[...]
error: (98, 'Address already in use')

And to confirm that it's TIME-WAIT that's stopping us:

[ponder ~]netstat -n | grep 12345
tcp 0 0 127.0.0.1:12345 127.0.0.1:49988 TIME_WAIT

If I change make_server_socket, then everything works fine.

In [8]: def make_server_socket():
...: sk = S.socket(S.AF_INET, S.SOCK_STREAM)
...: sk.setsockopt(S.SOL_SOCKET, S.SO_REUSEADDR, 1)
...: sk.bind(('', 12345))
...: sk.listen(5)
...: return sk
...:

In [10]: sk = make_server_socket()

In [11]: serve_client(sk)

In [13]: sk.close()

In [14]: sk = make_server_socket()

Done.

If you try this, note that both the old, closed socket /and/ the new one
must have SO_REUSEADDR set on them. If you try this interactively,
you'll have to wait for the non-SO_REUSEADDR socket to leave TIME-WAIT
before you can bind. But once you've done that, it'll work fine from
then on.

-- [mdw]