Inheritance from builtin list and override of methods.

[Carsten Haese]

And is there a mechanism in Python that will allow me to override
the operators of a class, for all its occurrences, even the ones
implemented on C built-in objects?

No.

For what it's worth, which is undoubtedly nothing,

Correct.

this is
something that I think needs to change. All this talk about new-style
classes and class-type unification is empty words if you can't override
anything on any type without having to know whether it's written in C or
Python.

Duncan's "No" response was not referring to overriding in general, it
was referring to the OP's original question which amounted to "Can I
affect the behavior of method Z by overriding methods X and Y".

The inability to influence a method by overriding completely different
methods has nothing to do with whether that method is implemented in C
or Python; it has to with whether the method in question calls the
overridden methods, and in general it won't.

You can change the behavior of a list's sort method by overriding sort.
You can't change the behavior of sort by overriding __getitem__ and
__setitem__, because sort does not call __getitem__ or __setitem__.

Hope this helps,

Carsten.

[Fredrik Lundh]

Why doesn't it?

because whoever wrote the class didn't do things that way, mostly for
efficiency reasons. there's nothing in Python that keeps you from using
template methods if you want, but that's a costly approach, so it's not
very common.

I suggest reading up on OO design patterns.

</F>

[Fredrik Lundh]

in general, methods on C objects are implemented in terms of operations
on the internal data structures, not in terms of a subset of the methods
already provided by the object.

But what is the reason that the assignment
l[2] = 6
call my function, but
l.append(3)

"obj[index] = value" maps to "obj.__setitem__(index, value)". reading
the documentation might help; start here:

http://docs.python.org/ref/specialnames.html

</F>

[Michalis Giannakidis]

Post by Fredrik Lundh
"obj[index] = value" maps to "obj.__setitem__(index, value)". reading
http://docs.python.org/ref/specialnames.html

In this documentation page it also says:
--snip---
then x[i] is equivalent3.2 to x.__getitem__(i).
--snip--
This, and other statements, are only roughly true for instances of new-style
classes.
--snip--

So which statements are actually true for new style classes?

Is l[0]=1 completely different from l.append(1) or maybe insert?

And is there a mechanism in Python that will allow me to override the
operators of a class, for all its occurrences, even the ones implemented on C
built-in objects?

[Fredrik Lundh]

Post by Michalis Giannakidis

Post by Fredrik Lundh
"obj[index] = value" maps to "obj.__setitem__(index, value)". reading
http://docs.python.org/ref/specialnames.html

--snip---
then x[i] is equivalent3.2 to x.__getitem__(i).
--snip--
This, and other statements, are only roughly true for instances of new-style
classes.
--snip--
So which statements are actually true for new style classes?
Is l[0]=1 completely different from l.append(1) or maybe insert?

l.append(1) is a method call. L[0]=1 is syntactic sugar for a
*different* method call. why is this so hard to understand?

</F>

[OKB (not okblacke)]

Post by Carsten Haese
You can change the behavior of a list's sort method by overriding
sort. You can't change the behavior of sort by overriding
__getitem__ and __setitem__, because sort does not call __getitem__
or __setitem__.

Why doesn't it?

[Michalis Giannakidis]

Post by Carsten Haese
You can change the behavior of a list's sort method by overriding
sort. You can't change the behavior of sort by overriding
__getitem__ and __setitem__, because sort does not call __getitem__
or __setitem__.

Why doesn't it?

I also expected that it did!

I perfectly understand that this adds significant penalty to the execution of
code. But in the way things are, I have to know ( or guess ?) how its
function has been implemented. And I cannot handle all cases the same.

If I provided my sort method for my class in Python, wouldn't this call my
__getitem__ and __setitem__ methods (considering l[i] = j assignment do)? Can
this be considered an in consitency?

[Fredrik Lundh]

Post by Michalis Giannakidis
I perfectly understand that this adds significant penalty to the execution of
code. But in the way things are, I have to know ( or guess ?) how its
function has been implemented.

and that's different from how object orientation usually works in
exactly what way?

</F>

[Carl Banks]

Post by Carsten Haese
You can change the behavior of a list's sort method by overriding
sort. You can't change the behavior of sort by overriding
__getitem__ and __setitem__, because sort does not call __getitem__
or __setitem__.

Why doesn't it?

Because the concerns of thousands of legitimate programmers who want
good performance out of their sorts outweigh the concerns of the one or
two hax0r d00ds who think it would be cool to hook into sort internals.


live-long-and-prosper-ly yr's,


Carl Banks

[Fredrik Lundh]

Post by Carl Banks
Because the concerns of thousands of legitimate programmers who want
good performance out of their sorts outweigh the concerns of the one or
two hax0r d00ds who think it would be cool to hook into sort internals.

... and haven't yet realized that using sorted() and converting back
from the resulting list will outperform *any* solution they can come
up with themselves.

</F>

[Duncan Booth]

Post by Michalis Giannakidis

Post by Fredrik Lundh
"obj[index] = value" maps to "obj.__setitem__(index, value)".
http://docs.python.org/ref/specialnames.html

--snip---
then x[i] is equivalent3.2 to x.__getitem__(i).
--snip--
This, and other statements, are only roughly true for instances of
new-style classes.
--snip--
So which statements are actually true for new style classes?

I think it means that the statements are true in general, but there may be
specific corner cases which break them. Usually you can assume that the
rough description will be good enough.

Post by Michalis Giannakidis
Is l[0]=1 completely different from l.append(1) or maybe insert?

Yes. It neither appends nor inserts.

Post by Michalis Giannakidis
And is there a mechanism in Python that will allow me to override the
operators of a class, for all its occurrences, even the ones
implemented on C built-in objects?

No.

[OKB (not okblacke)]

And is there a mechanism in Python that will allow me to override
the operators of a class, for all its occurrences, even the ones
implemented on C built-in objects?

No.

For what it's worth, which is undoubtedly nothing, this is
something that I think needs to change. All this talk about new-style
classes and class-type unification is empty words if you can't override
anything on any type without having to know whether it's written in C or
Python.

[Fredrik Lundh]

Could someone please explain the reasoning/behabiour of these?

in general, methods on C objects are implemented in terms of operations
on the internal data structures, not in terms of a subset of the methods
already provided by the object.

</F>

[Michalis Giannakidis]

in general, methods on C objects are implemented in terms of operations
on the internal data structures, not in terms of a subset of the methods
already provided by the object.

But what is the reason that the assignment
l[2] = 6
call my function, but
l.append(3)
doesn't ?

Also, why can't I override these functions (for example in order to get access
to the list.sort() internals)? Or can I?


#!/usr/bin/env python
class L(list):
def __getitem__(self, i):
print 'G', i
return list.__getitem__(self, i)
def __setitem__(self, i, y):
print 'S:', i, y
return list.__setitem__(self, i, y)
def __setslice__(self, i, j, y):
print 'SL:', i, j, y
return list.__setslice__(self, i, j, y)
def __iter__(self, x):
print 'iter:', x
return list.__iter__(self, x)

l = L()
l.append(3) # this does not call my __setitem__
l.append(2)
l.append(1)
l[2] = 6 # this calls my __setitem__
l.sort(key=lambda x: x )
print l

[Michalis Giannakidis]

Dear all

I tried to override methods of build in Python types, so as to change their
behavior.
I tried to override list.__getitem__ list.__setitem__ and some others alike
The test case is:

#!/usr/bin/env python
class L(list):
def __getitem__(self, i):
print 'G', i
return list.__getitem__(self, i)
def __setitem__(self, i, y):
print 'S:', i, y
return list.__setitem__(self, i, y)
def __setslice__(self, i, j, y):
print 'SL:', i, j, y
return list.__setslice__(self, i, j, y)
def __iter__(self, x):
print 'iter:', x
return list.__iter__(self, x)

l = L()
l.append(3) # this does not call my __setitem__
l.append(2)
l.append(1)
l[2] = 6 # this calls my __setitem__
l.sort(key=lambda x: x )
print l

I expected that the call to l.sort() would call my versions of the list
methods (iter, setitem etc ) but it didn't. Also the call to l.append didn't
call my setitem but the assignment l[2] = 6 did! I expected my versions of
iter, setitem, getitem to be called with the typical impementation of sorting
from C in mind.

Could someone please explain the reasoning/behabiour of these?

Python Version:
Python 2.4.2 (#1, May 26 2006, 14:35:35)
[GCC 3.3.6 (Gentoo 3.3.6, ssp-3.3.6-1.0, pie-8.7.8)] on linux2

Thank you very much in advance.

Michalis

[Carl Banks]

Could someone please explain the reasoning/behabiour of these?

in general, methods on C objects are implemented in terms of operations
on the internal data structures, not in terms of a subset of the methods
already provided by the object.

True, and it's also true for many Python classes. Here's a very silly
example:

class Cons(object):
def __init__(self,car,cdr):
self.car = car
self.cdr = cdr
def __getitem__(self,key):
if key == "car":
return self.car
if key == "cdr":
return self.cdr
raise KeyError(key)
def __getitem__(self,key,value):
if key == "car":
self.car = value
if key == "cdr":
self.cdr = value
raise KeyError(key)
def swap(self):
self.car,self.cdr = self.cdr,self.car

So hooking into __getitem__ and __setitem__ won't let you inpect the
swap operation. It's much the same with lists: they bypass __getitem__
and __setitem__ and go directly to their internal.

OTOH, I've written a few C types that deliberately go though slower
Python methods to allow overriding. But that has to be deliberate (for
the most part); as Frederick said, most C classes don't do that.


Carl Banks