How to get a directory file descriptor?

[Nick Craig-Wood]

Thanks for your reply. I checked my Python 2.5 install on Linux and there's the
O_DIRECTORY flag. However this is not mentioned anywhere in the Library Reference.
There's another question regarding to this flag though: I checked the manual of
O_DIRECTORY
If pathname is not a directory, cause the open to fail. This
flag is Linux-specific, and was added in kernel version 2.1.126,
to avoid denial-of-service problems if opendir(3) is called on a
FIFO or tape device, but should not be used outside of the
implementation of opendir(3).

That says avoid to me!

Note the "should not be used outside of the implementation of opendir(3)" part.
Does that mean using Python's os.open() with the O_DIRECTORY flag is a Bad
Thing?

IMHO yes.

In C, I think, calling opendir() followed by dirfd() should be the
correct way of doing this?

Here is how you do exactly that in python using ctypes

from ctypes import CDLL, c_char_p, c_int, Structure, POINTER
from ctypes.util import find_library

class c_dir(Structure):
"""Opaque type for directory entries, corresponds to struct DIR"""
c_dir_p = POINTER(c_dir)

c_lib = CDLL(find_library("c"))
opendir = c_lib.opendir
opendir.argtypes = [c_char_p]
opendir.restype = c_dir_p
dirfd = c_lib.dirfd
dirfd.argtypes = [c_dir_p]
dirfd.restype = c_int
closedir = c_lib.closedir
closedir.argtypes = [c_dir_p]
closedir.restype = c_int

dir_p = opendir(".")
print "dir_p = %r" % dir_p
dir_fd = dirfd(dir_p)
print "dir_fd = %r" % dir_fd
print "closed (rc %r)" % closedir(dir_p)

Which prints on my linux machine

dir_p = <ctypes.LP_c_dir object at 0xb7d13cd4>
dir_fd = 3
closed (rc 0)

I don't know why os doesn't wrap - opendir, closedir, dirfd, readdir
etc - I guess because except if you are doing something esoteric, then
os.list and os.walk do everything you could possibly want.

[greg]

import os
dirfd = os.open("directory-name", os.O_DIRECTORY)
os.fchdir(dirfd)

os.O_DIRECTORY must be fairly new -- it doesn't exist
in my 2.5 installation. But os.O_RDONLY seems to work
just as well for this purpose.

[alex23]

Thanks for your reply. I checked my Python 2.5 install on Linux and there's the
O_DIRECTORY flag. However this is not mentioned anywhere in the Library Reference.

Yes, I just noticed that myself.

? ? ? ?O_DIRECTORY
? ? ? ? ? ? ? If pathname is not a directory, cause the open ?to ?fail. ? This
? ? ? ? ? ? ? flag is Linux-specific, and was added in kernel version 2.1.126,

Well, that certainly explains it's absence :)

Note the "should not be used outside of the implementation of opendir(3)" part.
Does that mean using Python's os.open() with the O_DIRECTORY flag is a Bad
Thing?

I'm sorry, that's outside of my knowledge. But it seemed to work fine
when I tried it.

[Cong Ma]

Post by greg
os.O_DIRECTORY must be fairly new -- it doesn't exist
in my 2.5 installation. But os.O_RDONLY seems to work
just as well for this purpose.

Which OS are you using?
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Post by greg

import os
hasattr(os, 'O_DIRECTORY')

True
I'm pretty certain it was present under Windows XP as well.
--
http://mail.python.org/mailman/listinfo/python-list

Hello,

Thanks for your reply. I checked my Python 2.5 install on Linux and there's the
O_DIRECTORY flag. However this is not mentioned anywhere in the Library Reference.

There's another question regarding to this flag though: I checked the manual of
the Linux system call open(2), in which there's a paragraph on this flag:

O_DIRECTORY
If pathname is not a directory, cause the open to fail. This
flag is Linux-specific, and was added in kernel version 2.1.126,
to avoid denial-of-service problems if opendir(3) is called on a
FIFO or tape device, but should not be used outside of the
implementation of opendir(3).

Note the "should not be used outside of the implementation of opendir(3)" part.
Does that mean using Python's os.open() with the O_DIRECTORY flag is a Bad
Thing? In C, I think, calling opendir() followed by dirfd() should be the
correct way of doing this?

Thanks.

[greg]

Anyway, when I try to use O_RDONLY on Vista I get

According to the library ref, fchdir() is Unix-only, so there
probably isn't a use case for getting a file descriptor for
a directory on Windows in the first place.

[Steve Holden]

Post by greg
os.O_DIRECTORY must be fairly new -- it doesn't exist
in my 2.5 installation. But os.O_RDONLY seems to work
just as well for this purpose.

Which OS are you using?
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Post by greg

import os
hasattr(os, 'O_DIRECTORY')

True
I'm pretty certain it was present under Windows XP as well.

OK, here's a little mystery. This is Vista (spit):

Microsoft Windows [Version 6.0.6001]
Copyright (c) 2006 Microsoft Corporation. All rights reserved.

C:\Users\sholden>python
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Post by greg

import os
f = os.open(".", os.O_DIRECTORY)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'O_DIRECTORY'

Post by greg

hasattr(os, 'O_DIRECTORY')

False
Since these two are the exact same version I presume O_DIRECTORY is not
meaningful on Windows. Anyway, when I try to use O_RDONLY on Vista I get

Post by greg

f = os.open(os.path.abspath("Documents"), os.O_RDONLY)

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 13] Permission denied: 'C:\\Users\\sholden\\Documents'

regards
Steve

[Terry Reedy]

Post by Steve Holden

import os
hasattr(os, 'O_DIRECTORY')

True
I'm pretty certain it was present under Windows XP as well.

f = os.open(".", os.O_DIRECTORY)

File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'O_DIRECTORY'

The 3.0 manual lists 4 groups of flags: unix and windows, unix only,
windows only, gnu only (check system-specific manual). O_DIRECTORY is
in the gnu group.

[alex23]

Post by greg
os.O_DIRECTORY must be fairly new -- it doesn't exist
in my 2.5 installation. But os.O_RDONLY seems to work
just as well for this purpose.

Which OS are you using?

Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Post by greg

import os
hasattr(os, 'O_DIRECTORY')

True

I'm pretty certain it was present under Windows XP as well.

[alex23]

Post by Steve Holden

I'm pretty certain it was present under Windows XP as well.

Since these two are the exact same version I presume O_DIRECTORY is not
meaningful on Windows. Anyway, when I try to use O_RDONLY on Vista I get

Sorry, my bad. I got the exact same "Permission denied" response when
I first tried this under XP, so I shelled to a linux box and tested
there...then promptly forgot all about it. In my defence, it was
pretty late at night when I did so :)

In fact, I can't even find a reference to O_DIRECTORY in the manual at
all...

[Cong Ma]

Dear all,

Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?

Thanks for your reply.

Regards,
Cong.

[Baptiste Carvello]

Post by Cong Ma
Dear all,
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?
Thanks for your reply.
Regards,
Cong.
--
http://mail.python.org/mailman/listinfo/python-list

Hello,

I believe you want os.open :

Python 2.5.2 (r252:60911, Sep 29 2008, 21:10:35)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Post by Cong Ma

import os
d=os.open('mydir',os.O_RDONLY)
d

3

Post by Cong Ma

os.fchdir(d)
os.getcwd()

'/home/baptiste/mydir'
Cheers,
Baptiste

[Cong Ma]

Is this what you want?
ofiles = [open(x) for x in os.listdir(os.getcwd())]

'open' returns a "file object", whereas the OP is after "file
descriptors", which are returned by 'os.open'.
--
http://mail.python.org/mailman/listinfo/python-list

You got it.

So if I can't seem to get directory file descriptors, the only way to use
os.fchdir() in Python is to embed Python in C code?

[Cong Ma]

Post by Nick Craig-Wood
Here is how you do exactly that in python using ctypes
from ctypes import CDLL, c_char_p, c_int, Structure, POINTER
from ctypes.util import find_library
"""Opaque type for directory entries, corresponds to struct DIR"""
c_dir_p = POINTER(c_dir)
c_lib = CDLL(find_library("c"))
opendir = c_lib.opendir
opendir.argtypes = [c_char_p]
opendir.restype = c_dir_p
dirfd = c_lib.dirfd
dirfd.argtypes = [c_dir_p]
dirfd.restype = c_int
closedir = c_lib.closedir
closedir.argtypes = [c_dir_p]
closedir.restype = c_int
dir_p = opendir(".")
print "dir_p = %r" % dir_p
dir_fd = dirfd(dir_p)
print "dir_fd = %r" % dir_fd
print "closed (rc %r)" % closedir(dir_p)
Which prints on my linux machine
dir_p = <ctypes.LP_c_dir object at 0xb7d13cd4>
dir_fd = 3
closed (rc 0)
I don't know why os doesn't wrap - opendir, closedir, dirfd, readdir
etc - I guess because except if you are doing something esoteric, then
os.list and os.walk do everything you could possibly want.

Thank you so much Nick. I'll take a look into the ctypes module.

I guess the reason of Python library not including those wrappers has something
to do with standard compliance. The fchdir(2) manual says "comforming to
POSIX.1-2001" but the opendir(3) manual says "fdopendir() is specified in
POSIX.1-2008."

[D'Arcy J.M. Cain]

On Tue, 25 Nov 2008 16:02:56 +0800

Post by Cong Ma
Can you give me some hint on getting a directory file descriptor in Python?

print each

Your code fetches a bunch of strings representing file names in the working
directory, which is fine. But what I want is something like an integer file
descriptor, like the one returned by os.open() for files, or the Linux dirfd()
call, which returns an integer for a pointer to a DIR stream.

Is this what you want?

ofiles = [open(x) for x in os.listdir(os.getcwd())]

[r0g]

Post by Cong Ma
Dear all,
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?
Thanks for your reply.
Regards,
Cong.

for each in os.listdir(os.getcwd()):
print each

Roger.

[alex23]

Post by Cong Ma
So if I can't seem to get directory file descriptors, the only way to use
os.fchdir() in Python is to embed Python in C code?

This doesn't work for you?

import os
dirfd = os.open("directory-name", os.O_DIRECTORY)
os.fchdir(dirfd)

Are you getting errors? Incorrect behaviour?

[r0g]

Post by Cong Ma
Dear all,
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?
Thanks for your reply.
Regards,
Cong.

print each
Roger.
--
http://mail.python.org/mailman/listinfo/python-list

Roger,
It seemed I didn't make it clearly enough...
Your code fetches a bunch of strings representing file names in the working
directory, which is fine. But what I want is something like an integer file
descriptor, like the one returned by os.open() for files, or the Linux dirfd()
call, which returns an integer for a pointer to a DIR stream.
Regards,
Cong.

Erm, yeah, that's the os.listdir() bit.
Roger.

Actually, just re-reading your last mail, it isn't what you want at all
is it?! Sorry!

Roger.

[r0g]

Post by Cong Ma
Dear all,
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?
Thanks for your reply.
Regards,
Cong.

print each
Roger.
--
http://mail.python.org/mailman/listinfo/python-list

Roger,
It seemed I didn't make it clearly enough...
Your code fetches a bunch of strings representing file names in the working
directory, which is fine. But what I want is something like an integer file
descriptor, like the one returned by os.open() for files, or the Linux dirfd()
call, which returns an integer for a pointer to a DIR stream.
Regards,
Cong.

Erm, yeah, that's the os.listdir() bit.

Roger.

[Cong Ma]

Post by Cong Ma
Dear all,
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?
Thanks for your reply.
Regards,
Cong.

print each
Roger.
--
http://mail.python.org/mailman/listinfo/python-list

Roger,

It seemed I didn't make it clearly enough...

Your code fetches a bunch of strings representing file names in the working
directory, which is fine. But what I want is something like an integer file
descriptor, like the one returned by os.open() for files, or the Linux dirfd()
call, which returns an integer for a pointer to a DIR stream.

Regards,
Cong.

[alex23]

Is this what you want?
ofiles = [open(x) for x in os.listdir(os.getcwd())]

'open' returns a "file object", whereas the OP is after "file
descriptors", which are returned by 'os.open'.

[alex23]

Post by Cong Ma
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?

Try: os.open('<dirname>', os.O_RDONLY)
Check os for a list of the valid flag types.

Sorry, that should probably be os.O_DIRECTORY.

[alex23]

Post by Cong Ma
Can you give me some hint on getting a directory file descriptor in Python?
Besides, what's good about os.fchdir() if I can't get a directory fd in the
first place?

Try: os.open('<dirname>', os.O_RDONLY)

Check os for a list of the valid flag types.