file names longer than MAX_PATH under Windows 2003

[Steven D'Aprano]

Hello.
E:\files\..................\something.dat
On MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/naming_a_file.asp) described method to access
files with path length
up to 32000 bytes: just add prefix \\?\ to file name.
But when I try to pass prefixed name to file(), I get the same result as when I don't add the prefix: file not found. May be Python
just doesn't support long unicode filenames?
Is there way to open such files?

Backslashes have special meaning to Python and need to be escaped. If you
f = file("E:\files\...\something.dat", "r")
Python's string escape rules means you are actually trying to open the
file "E:files...something.dat" which doesn't exist.

[slaps head]
I seem to be a bit confused about string escaping rules. Only some
backslashes have special meaning, the rest remain in the string.

Sergey, you said UNICODE file names. Not just ordinary strings.

Are you passing a unicode object to the function?

f = file(u"E:\\files\\...\\something.dat", "r")

[Sergey]

"Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in message news:mailman.1895.1139917457.27775.python-list at python.org...
[Sergey]

But note that r prefix to the string. Is it possible
that your string didn't include it? If not, then the
backslash character which Windows uses as a separator
can be stolen by Python which sees it as an escaping
character.

print c

\\?\e:\files\........

[Sergey]

Post by Steven D'Aprano
Are you passing a unicode object to the function?
f = file(u"E:\\files\\...\\something.dat", "r")

c

u'\\\\.\\e:\\files\\\u041f\u0420\u041e\u0414\u041e \u041c\u0435\u043d\u0435\u043
[many unicode chars skipped]
44b\u0439_\u0411\u044e\u0434\u0436\u0435\u0442.xls'

[Sergey]

Hello.

I try to open file with pathname length 282 bytes:
E:\files\..................\something.dat

On MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/naming_a_file.asp) described method to access
files with path length
up to 32000 bytes: just add prefix \\?\ to file name.
But when I try to pass prefixed name to file(), I get the same result as when I don't add the prefix: file not found. May be Python
just doesn't support long unicode filenames?
Is there way to open such files?

[Sergey]

f = file("E:/files/.../something.dat", "r")
Does this help?

backslashes quoted.

print c

\\.\e:\files\xxxxxxxxxxxxxxxx\xxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxx\xxxxxxxxxxx xxxxxxxxx
\xxxxxxxxxxxxxxxxxxxxxxx\xxxxxxxxxxxxxx\xxxxxxxxxxxxxxxxxxxxx\xxxxxxxxxxxx
xxxxxxxxxxxxxxx\xxxxxxxxxxxxxxxxxxxxxxxxx\xxxxxxxxxxxxxxxxxxxxxxx\xxxxxx xxxxxxxxxx\xxxxxxxxxxxxxxxxxxxxxxxxxxxx.xls

(cyrillic letters in filename here I replaced with x-es)

file(c,"rb")

Traceback (most recent call last):
File "<stdin>", line 1, in ?
IOError: [Errno 2] No such file or directory: u'\\\\.\\e:\\files\\\u041f\u0420....filename skipped....\u0442.xls'

win32file.CreateFile(c,win32file.GENERIC_READ,0,None,win32file.OPEN_EXISTING

,0,None)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
pywintypes.error: (206, 'CreateFile', 'The filename or extension is too long.')

When I open explorer, I can browse folder and see this file (but I can do almost nothing with it). It was created through share.
Ntbackup can work with the file - why Python can't?

[Steven D'Aprano]

Hello.
E:\files\..................\something.dat
On MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/naming_a_file.asp) described method to access
files with path length
up to 32000 bytes: just add prefix \\?\ to file name.
But when I try to pass prefixed name to file(), I get the same result as when I don't add the prefix: file not found. May be Python
just doesn't support long unicode filenames?
Is there way to open such files?

Backslashes have special meaning to Python and need to be escaped. If you
do this:

f = file("E:\files\...\something.dat", "r")

Python's string escape rules means you are actually trying to open the
file "E:files...something.dat" which doesn't exist.

You should escape the backslashes:

f = file("E:\\files\\...\\something.dat", "r")

or use raw strings:

f = file(r"E:\files\...\something.dat", "r")

or just use forward slashes and let Windows deal with it:

f = file("E:/files/.../something.dat", "r")


Does this help?

[Tim Golden]

[Sergey]

| I try to open file with pathname length 282 bytes:
| E:\files\..................\something.dat

| [... MS advise ...] just add prefix \\?\ to file name.
| But when I try to pass prefixed name to file(), I get the
| same result as when I don't add the prefix: file not found.

With a file called c:\temp\test.txt, I successfully
opened and read it like this:

print open (r"\\?\C:\temp\test.txt").read ()

So the basic functionality works. I didn't
artificially generate a long path to see if
there's a problem in that direction.

But note that r prefix to the string. Is it possible
that your string didn't include it? If not, then the
backslash character which Windows uses as a separator
can be stolen by Python which sees it as an escaping
character.

TJG

________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

[Sergey]

"Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in message news:mailman.1900.1139925147.27775.python-list at python.org...
[Sergey]

I see from another post that CreateFile cannot open your file.
That puts it further away from Python, although it doesn't
explain how some other program can see the files. Can you use
os.startfile (or its equivalent win32api.ShellExecute from
pywin32)? Perhaps if you were to chdir to the directory in
question you'd be able to access the file.

I found error in filename (extra backslash) and now I can open file.
But another trouble appeared: I cannot listdir so long name.
I examined posixmodule.c... There are wcsncpy with hard limit of MAX_PATH*2+5.
So... RIP, my module...

[Claudio Grondi]

Post by Sergey
[Sergey]

I see from another post that CreateFile cannot open your file.
That puts it further away from Python, although it doesn't
explain how some other program can see the files. Can you use
os.startfile (or its equivalent win32api.ShellExecute from
pywin32)? Perhaps if you were to chdir to the directory in
question you'd be able to access the file.

I found error in filename (extra backslash) and now I can open file.
But another trouble appeared: I cannot listdir so long name.
I examined posixmodule.c... There are wcsncpy with hard limit of MAX_PATH*2+5.
So... RIP, my module...

I don't know if and how it apply and can be of any help here, but in my
C programs in the very past after switching from DOS to Windows long
names a following trick solved often my problems with too long names:
step 1: setting the current directory to the beginning part of the
name with a length which can be passed as a function parameter
step 2: usage of the remaining part of name as name passed as
parameter to the file accessing function (relative to current directory)

Claudio

[Sergey]

I don't know if and how it apply and can be of any help here, but in my C programs in the very past after switching from DOS to
step 1: setting the current directory to the beginning part of the name with a length which can be passed as a function
parameter
step 2: usage of the remaining part of name as name passed as parameter to the file accessing function (relative to current
directory)

It seems this method doesn't help here.

[Tim Golden]

[Sergey]

| "Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in
| message news:mailman.1900.1139925147.27775.python-list at python.org...
| [Sergey]
|
| >I see from another post that CreateFile cannot open your file.
| >That puts it further away from Python, although it doesn't
| >explain how some other program can see the files. Can you use
| >os.startfile (or its equivalent win32api.ShellExecute from
| >pywin32)? Perhaps if you were to chdir to the directory in
| >question you'd be able to access the file.
|
| I found error in filename (extra backslash) and now I can open file.
| But another trouble appeared: I cannot listdir so long name.
| I examined posixmodule.c... There are wcsncpy with hard limit
| of MAX_PATH*2+5.
| So... RIP, my module...

Have a look at win32file.FindFilesIterator from the pywin32 extensions.
Maybe that can cope? (I haven't looked at the source).

<code>
import win32file
for (
attr, created, accessed, written, size_hi, size_lo,
_, _,
filename, alt_filename
) in win32file.FindFilesIterator ("c:/temp/*"):
print filename

</code>

TJG

________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

[&quot;Martin v. Löwis&quot;]

(but now I must have two pieces of code, one for linux and one for windows)
Interesting, why developers of python didn't use here all power of win32 API?

It will in Python 2.5.

Regards,
Martin

[Sergey]

"Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in message news:mailman.1897.1139918374.27775.python-list at python.org...
[Sergey]

Not to state the obvious, but can you cut-and-paste that long
string (the one starting with \\?\e:\...) from the Python
interpreter into the [S]tart [R]un [O]pen field to see what
comes up? I'm just trying to make sure of the most straightforward
fact: that the file you've got there definitely does exist!

I cannot do this: when I paste filename there, trail of filename is missing due to length limit in input line.
But I strongly suppose that file exists as its name was get through os.listdir.

[Sergey]

"Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in message news:mailman.1897.1139918374.27775.python-list at python.org...
[Sergey]

Not to state the obvious, but can you cut-and-paste that long
string (the one starting with \\?\e:\...) from the Python
interpreter into the [S]tart [R]un [O]pen field to see what
comes up? I'm just trying to make sure of the most straightforward
fact: that the file you've got there definitely does exist!

Sorry, it was my error - there was extra backslash in the middle of path.
\\?\path works OK.

[Sergey]

"Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in message news:mailman.1956.1140020466.27775.python-list at python.org...
[Sergey]

Post by Tim Golden
Have a look at win32file.FindFilesIterator from the pywin32 extensions.
Maybe that can cope? (I haven't looked at the source).

Yeah, it works!
THANK YOU!
(but now I must have two pieces of code, one for linux and one for windows)
Interesting, why developers of python didn't use here all power of win32 API?

[Tim Golden]

[Sergey]

| "Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in
| message news:mailman.1897.1139918374.27775.python-list at python.org...
| [Sergey]
|
| >Not to state the obvious, but can you cut-and-paste that long
| >string (the one starting with \\?\e:\...) from the Python
| >interpreter into the [S]tart [R]un [O]pen field to see what
| >comes up? I'm just trying to make sure of the most straightforward
| >fact: that the file you've got there definitely does exist!
|
| I cannot do this: when I paste filename there, trail of
| filename is missing due to length limit in input line.
| But I strongly suppose that file exists as its name was get
| through os.listdir.

I see from another post that CreateFile cannot open your file.
That puts it further away from Python, although it doesn't
explain how some other program can see the files. Can you use
os.startfile (or its equivalent win32api.ShellExecute from
pywin32)? Perhaps if you were to chdir to the directory in
question you'd be able to access the file.

Don't know really; clutching at straws.

TJG

________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________

[Tim Golden]

[Sergey]

| "Tim Golden" <tim.golden at viacom-outdoor.co.uk> wrote in
| message news:mailman.1895.1139917457.27775.python-list at python.org...
| [Sergey]
|
| >But note that r prefix to the string. Is it possible
| >that your string didn't include it? If not, then the
| >backslash character which Windows uses as a separator
| >can be stolen by Python which sees it as an escaping
| >character.
|
| It's ok with backslashes, I don't use r'', but double them so
| string is correct:
| >>> print c
| \\?\e:\files\........

Not to state the obvious, but can you cut-and-paste that long
string (the one starting with \\?\e:\...) from the Python
interpreter into the [S]tart [R]un [O]pen field to see what
comes up? I'm just trying to make sure of the most straightforward
fact: that the file you've got there definitely does exist!

TJG

________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________