python.exe vs pythonw.exe difference?

[Emile van Sebille]

[Thomas]

I get exactly the same, on Win XP Pro, both for sys.stdout and
sys.stderr.

That's good to know! Thanks.

...
Since it seems XP shows the same behaviour than win98SE, has the
behaviour of Python changed? Were IOErrors ignored on sys.stdout

or

sys.stderr in earlier versions?

No, the same program fails the same way here under pythonw 2.2.3 and

2.1.3

(with s/file/open/ and s/True/1/). The OP wasn't clear about what

"it"

meant, though (in "it used to work"). I guess it's most likely he

meant

running Zope 2.5 as a service used to work, not that running Zope

2.5 by

hand from a DOS box with pythonw used to work, but don't know. It's
certainly possible that something relevant changed in Zope, and/or

in how

Zope tries to live with Windows services.

To clarify, at that site I'm migrating from zope2.5/python2.1/win2k
box to zope2.7/python2.3/winxppro. When "it used to work" I was
referring to my product running within zope as a service. For initial
debugging in the new environment, I started in console mode. Once
things stabilized, I started the service up and the problem surfaced.
Backtracking through the code I found the service starting the
application as per my earlier post with pythonw.exe which led me to
the direct comparison.

It did look like there'd been some significant refactoring going on
within zope and windows service interaction, but replicating the
effect directly using pythonw vs python seems to take that out of the
picture. (hence the post to the python list and not the zope list. I
really should remember that python-dev is an open list now...)

At this point, I won't be back to that site before next week, although
I may take some time to test this weekend. It sounds like I should
rework any areas that spew output to the console. Is there something
better than checking os.path.basename(sys.executable)?

ideas-ain't-code-ly y'rs - tim

That'd be too easy! Just ask that other Tim...

Emile van Sebille
emile at fenx dot com
emile at fenx.com

[Bengt Richter]

[Thomas Heller]

[I'm currently reading python-list via the gmane nntp interface, I
don't know whether there really is a gmane.comp.web.zope.devel
newsgroup]

I don't know either, so I'll copy you directly.
[Tim]

...
"""
import sys
if 1: # edit to 1 for stdout, 0 for stderr
console = sys.stdout
console = sys.stderr
import traceback
tb = file('tb.txt', 'w')
i = 0
i += 1
console.write('.')
print >> tb, "Died when trying to write byte", i
traceback.print_exc(file=tb)
tb.close()
"""
Under Win98SE, and regardless of whether it writes to stdout or
stderr, it dies when run under pythonw, and tb.txt contains this
Died when trying to write byte 4097
File "wr.py", line 14, in ?
console.write('.')
IOError: [Errno 9] Bad file descriptor

[Thomas]

I get exactly the same, on Win XP Pro, both for sys.stdout and
sys.stderr.

That's good to know! Thanks.

...
Since it seems XP shows the same behaviour than win98SE, has the
behaviour of Python changed? Were IOErrors ignored on sys.stdout or
sys.stderr in earlier versions?

No, the same program fails the same way here under pythonw 2.2.3 and 2.1.3
(with s/file/open/ and s/True/1/). The OP wasn't clear about what "it"
meant, though (in "it used to work"). I guess it's most likely he meant
running Zope 2.5 as a service used to work, not that running Zope 2.5 by
hand from a DOS box with pythonw used to work, but don't know. It's
certainly possible that something relevant changed in Zope, and/or in how
Zope tries to live with Windows services.

IIRC, /F first proposed that pythonw.exe should create a console to
have a place to show tracebacks. Sounds like a good idea to me.

Some way of having pythonw not drop output into the bit bucket has sounded
like a good idea to everyone for about a decade now <wink>.
ideas-ain't-code-ly y'rs - tim

You can call AllocConsole safely at the last moment before spewing text, I think,
and without even checking if there already is one (I think it will quietly do nothing
if there is, IIRC from old Delphi days).

I don't think people want to see console windows along with their GUIs when things are normal,
but post mortem is another thing, and this is a simple call. Also it gives you a handy separate
window to put debug messages to just by using print, which can be handy with
if __debug__: -conditional code or temporary hacks.

_____________________________________________________________________________

The AllocConsole function allocates a new console for the
calling process.


BOOL AllocConsole(VOID)


Parameters


This function has no parameters.


Return Value


If the function succeeds, the return value is TRUE. If the
function fails, the return value is FALSE. To get extended
error information, call GetLastError.


Remarks


A process can only be associated with one console, so
AllocConsole fails if the calling process already has a
console. A process can use the FreeConsole function to
detach itself from its current console, and then it can call
AllocConsole to create a new console. If the calling process
creates a child process, the child inherits the new console.
AllocConsole also sets up standard input, standard output,
and standard error handles for the new console. The standard
input handle is a handle to the console's input buffer, and
the standard output and standard error handles are handles
to the console's screen buffer. To retrieve these handles,
use the GetStdHandle function.


This function is primarily used by graphics applications to
create a console window. Graphics applications are
initialized without a console. Console applications are
normally initialized with a console, unless they are created
as detached processes (by calling the CreateProcess function
with the DETACHED_PROCESS flag).


See Also


CreateProcess, FreeConsole, GetStdHandle
_____________________________________________________________________________

AllocConsole should be easy to call through one of the win32 interfaces.
Maybe we should just make a DLL that would just do that and nothing else when imported.

Re Emile's situation, I think there may be an additional factor. I.e., output from a windows
service to an interactive window requires a special checkbox somewhere in the service setup,
I think, since services normally run as a non-interactive process with separate user identity
from whoever might be locally logged on.

That bit in your '.' output suggests to me that there's a 4k buffer write being triggered and
that is the thing that runs into no place to write. I wonder if it wouldn't trigger earlier with
newlines or explicit flush or -U ?

HTH

Regards,
Bengt Richter

[Tim Peters]

[Thomas Heller]

[I'm currently reading python-list via the gmane nntp interface, I
don't know whether there really is a gmane.comp.web.zope.devel
newsgroup]

I don't know either, so I'll copy you directly.

[Tim]

...
"""
import sys
if 1: # edit to 1 for stdout, 0 for stderr
console = sys.stdout
console = sys.stderr
import traceback
tb = file('tb.txt', 'w')
i = 0
i += 1
console.write('.')
print >> tb, "Died when trying to write byte", i
traceback.print_exc(file=tb)
tb.close()
"""
Under Win98SE, and regardless of whether it writes to stdout or
stderr, it dies when run under pythonw, and tb.txt contains this
Died when trying to write byte 4097
File "wr.py", line 14, in ?
console.write('.')
IOError: [Errno 9] Bad file descriptor

[Thomas]

I get exactly the same, on Win XP Pro, both for sys.stdout and
sys.stderr.

That's good to know! Thanks.

...
Since it seems XP shows the same behaviour than win98SE, has the
behaviour of Python changed? Were IOErrors ignored on sys.stdout or
sys.stderr in earlier versions?

No, the same program fails the same way here under pythonw 2.2.3 and 2.1.3
(with s/file/open/ and s/True/1/). The OP wasn't clear about what "it"
meant, though (in "it used to work"). I guess it's most likely he meant
running Zope 2.5 as a service used to work, not that running Zope 2.5 by
hand from a DOS box with pythonw used to work, but don't know. It's
certainly possible that something relevant changed in Zope, and/or in how
Zope tries to live with Windows services.

IIRC, /F first proposed that pythonw.exe should create a console to
have a place to show tracebacks. Sounds like a good idea to me.

Some way of having pythonw not drop output into the bit bucket has sounded
like a good idea to everyone for about a decade now <wink>.

ideas-ain't-code-ly y'rs - tim

[Thomas Heller]

[Thomas Heller]

[I'm currently reading python-list via the gmane nntp interface, I
don't know whether there really is a gmane.comp.web.zope.devel
newsgroup]

I don't know either, so I'll copy you directly.

It seems to exist, but I don't read it - so this is a good idea.

[Tim]

[script, writing to sys.stderr and/or sys.stdout]

Under Win98SE, and regardless of whether it writes to stdout or
stderr, it dies when run under pythonw, and tb.txt contains this
Died when trying to write byte 4097
File "wr.py", line 14, in ?
console.write('.')
IOError: [Errno 9] Bad file descriptor

[Thomas]

I get exactly the same, on Win XP Pro, both for sys.stdout and
sys.stderr.

That's good to know! Thanks.

...
Since it seems XP shows the same behaviour than win98SE, has the
behaviour of Python changed? Were IOErrors ignored on sys.stdout or
sys.stderr in earlier versions?

No, the same program fails the same way here under pythonw 2.2.3 and 2.1.3
(with s/file/open/ and s/True/1/). The OP wasn't clear about what "it"
meant, though (in "it used to work"). I guess it's most likely he meant
running Zope 2.5 as a service used to work, not that running Zope 2.5 by
hand from a DOS box with pythonw used to work, but don't know. It's
certainly possible that something relevant changed in Zope, and/or in how
Zope tries to live with Windows services.

IIRC, /F first proposed that pythonw.exe should create a console to
have a place to show tracebacks. Sounds like a good idea to me.

Some way of having pythonw not drop output into the bit bucket has sounded
like a good idea to everyone for about a decade now <wink>.

That may be, and the idea sounds even better if pythonw 'crashes' when
the bit bucket is full.

ideas-ain't-code-ly y'rs - tim

Sure ;-)

Thomas

[Tim Peters]

[Emile van Sebille]

Post by Emile van Sebille
...
At this point, I won't be back to that site before next week, although
I may take some time to test this weekend. It sounds like I should
rework any areas that spew output to the console. Is there something
better than checking os.path.basename(sys.executable)?

You could use the logging module all the time instead of trying to make the
destination conditional on the executable name. Bad errors should probably
be reported to the Windows Event Log service too; doing a serious job of
making a thing a Windows Service is painful.

[Emile van Sebille]

Dear group,

I've possibly narrowed a problem I'm having running zope as a service
on winxp pro sp 1 in that when started from a command line as:

c:\zope\v27\lib\python\python.exe
c:\zope\v27\lib\python\zope\startup\run.py -C
c:\zope\v27\instance\etc\zope.conf

it starts up just fine (although now running from the console).

But when I start it with:

c:\zope\v27\lib\python\pythonw.exe
c:\zope\v27\lib\python\zope\startup\run.py -C
c:\zope\v27\instance\etc\zope.conf

it dies after about 30 seconds.

It wouldn't surprise me that I'm doing something it doesn't like (I'm
spawning additional processes from within a product but it worked fine
with 2.5), but I'm somewhat at a loss as to debugging it in that when
run as a console app it works fine, but when run windowless it
doesn't.

Do I have to write out check points to a file? or is there some way to
use Mark Hammonds process debugging tools? Or is this a bug, known or
otherwise?

Pointers and hints welcome.

Emile van Sebille
emile at fenx dot com
emile at fenx.com

[Thomas Heller]

Post by Emile van Sebille
Dear group,
I've possibly narrowed a problem I'm having running zope as a service
c:\zope\v27\lib\python\python.exe
c:\zope\v27\lib\python\zope\startup\run.py -C
c:\zope\v27\instance\etc\zope.conf
it starts up just fine (although now running from the console).
c:\zope\v27\lib\python\pythonw.exe
c:\zope\v27\lib\python\zope\startup\run.py -C
c:\zope\v27\instance\etc\zope.conf
it dies after about 30 seconds.
It wouldn't surprise me that I'm doing something it doesn't like (I'm
spawning additional processes from within a product but it worked fine
with 2.5), but I'm somewhat at a loss as to debugging it in that when
run as a console app it works fine, but when run windowless it
doesn't.
Do I have to write out check points to a file? or is there some way to
use Mark Hammonds process debugging tools? Or is this a bug, known or
otherwise?
Pointers and hints welcome.

It has been reported that writing to the original sys.stdout (and maybe
also sys.stderr) sooner or later raises an IOError when running
pythonw.exe, unless these are redirected. Could this be the problem?

Thomas

[Tim Peters]

[moving this from comp.lang.python to zope-dev; it really belongs on a
Zope list, although schemes to change what pythonw does probably belong
on python-dev]

[Emile van Sebille <emile at fenx.com>]

Post by Emile van Sebille
I've possibly narrowed a problem I'm having running zope as a service
c:\zope\v27\lib\python\python.exe \

c:\zope\v27\lib\python\zope\startup\run.py -C \
c:\zope\v27\instance\etc\zope.conf

[tim inserted backslashes above, to make the line structure clear]

Post by Emile van Sebille
it starts up just fine (although now running from the console).
c:\zope\v27\lib\python\pythonw.exe \

c:\zope\v27\lib\python\zope\startup\run.py -C \
c:\zope\v27\instance\etc\zope.conf

Post by Emile van Sebille
it dies after about 30 seconds.
It wouldn't surprise me that I'm doing something it doesn't like (I'm
spawning additional processes from within a product but it worked
fine with 2.5),

It's unclear what "it" means, in "it wworked fine with 2.5". For example,
do you mean that the second command line, using pythonw.exe explicitly from
a DOS box worked fine, or do you mean that running Zope as a service on XP
Pro SP1 worked fine, or ...?

Post by Emile van Sebille
but I'm somewhat at a loss as to debugging it in> that when run as a
console app it works fine, but when run windowless it doesn't.

Did you look in your Zope log file(s) for tracebacks?

Post by Emile van Sebille
Do I have to write out check points to a file? or is there some way
to use Mark Hammonds process debugging tools? Or is this a bug,
known or otherwise?

[Thomas Heller]

It has been reported that writing to the original sys.stdout (and
maybe also sys.stderr) sooner or later raises an IOError when running
pythonw.exe, unless these are redirected. Could this be the problem?

It could, although I have no idea what WinXP does (and don't have access to
XP). Here's a Python program to try:

"""
import sys
if 1: # edit to 1 for stdout, 0 for stderr
console = sys.stdout
else:
console = sys.stderr

import traceback
tb = file('tb.txt', 'w')

try:
i = 0
while True:
i += 1
console.write('.')
except:
print >> tb, "Died when trying to write byte", i
traceback.print_exc(file=tb)
tb.close()
"""

Under Win98SE, and regardless of whether it writes to stdout or stderr, it
dies when run under pythonw, and tb.txt contains this after:

Died when trying to write byte 4097
Traceback (most recent call last):
File "wr.py", line 14, in ?
console.write('.')
IOError: [Errno 9] Bad file descriptor

The point of pythonw.exe is that no console is created or inherited, and the
default stdin, stdout and stderr provided by MS C in that case are unusable
(although the output flavors can appear to be usable until some secret MS
limit is exceeded -- at least under Win98SE).

[Tres Seaver]

Post by Tim Peters
[moving this from comp.lang.python to zope-dev; it really belongs on a
Zope list, although schemes to change what pythonw does probably belong
on python-dev]

[I'm currently reading python-list via the gmane nntp interface, I don't
know whether there really is a gmane.comp.web.zope.devel newsgroup]

Heh, yep. That is where I saw yours.

Tres.

[Thomas Heller]

Post by Tim Peters
[moving this from comp.lang.python to zope-dev; it really belongs on a
Zope list, although schemes to change what pythonw does probably belong
on python-dev]

[I'm currently reading python-list via the gmane nntp interface, I don't
know whether there really is a gmane.comp.web.zope.devel newsgroup]

Post by Tim Peters
[Thomas Heller]

It has been reported that writing to the original sys.stdout (and
maybe also sys.stderr) sooner or later raises an IOError when running
pythonw.exe, unless these are redirected. Could this be the problem?

It could, although I have no idea what WinXP does (and don't have access to
"""
import sys
if 1: # edit to 1 for stdout, 0 for stderr
console = sys.stdout
console = sys.stderr
import traceback
tb = file('tb.txt', 'w')
i = 0
i += 1
console.write('.')
print >> tb, "Died when trying to write byte", i
traceback.print_exc(file=tb)
tb.close()
"""
Under Win98SE, and regardless of whether it writes to stdout or stderr, it
Died when trying to write byte 4097
File "wr.py", line 14, in ?
console.write('.')
IOError: [Errno 9] Bad file descriptor

I get exactly the same, on Win XP Pro, both for sys.stdout and sys.stderr.

Post by Tim Peters
The point of pythonw.exe is that no console is created or inherited, and the
default stdin, stdout and stderr provided by MS C in that case are unusable
(although the output flavors can appear to be usable until some secret MS
limit is exceeded -- at least under Win98SE).

Since it seems XP shows the same behaviour than win98SE, has the
behaviour of Python changed? Were IOErrors ignored on sys.stdout or
sys.stderr in earlier versions?

IIRC, /F first proposed that pythonw.exe should create a console to have
a place to show tracebacks. Sounds like a good idea to me.

Thomas