Overloading operators for currying, a PEP309 suggestion

Discussion:

Stephen Horne

2003-03-11 14:50:35 UTC

On 11 Mar 2003 15:38:41 +0100, Bernhard Herzog <bh at intevation.de>

I would prefer defining the __len__, __getitem__, __setitem__ and
__delitem__ methods to allow direct manipulation (and creation and
deletion) of the closure.

That would mean that the length of a function with no arguments is 0,
right?

Probably

That's bad and will likely introduce subtle bugs because then all
functions with no arguments are suddenly considered false.

Why would a function object be treated as a boolean?

What about other callable objects? Classes, methods or instances that
have a __call__ method (They might even implement it through
__getattr__)?

Good point.

Lenard Lindstrom

2003-03-12 07:08:01 UTC

Permalink

"Stephen Horne" <intentionally at blank.co.uk> wrote in message

You may be onto something, but I'm not keen on your '|' version in
particular.

I chose the '|' bitwise or and '<<' left shift operators because of a
superficial simularity between currying a parameter list and manipulating a
string of bits. Doing an 'or' allows you to selectively set arguments; doing
a 'shift' allows you to remove arguments sequencially. However, there are
many operators to choose from.

I would prefer defining the __len__, __getitem__, __setitem__ and
__delitem__ methods to allow direct manipulation (and creation and
deletion) of the closure. Though probably the function object should
be immutable.

Yes, I think the curried function object should be immutable. This leaves
out __setitem__. But is it needed?

How about...
<function-obj> [ <int> | <slice> ] -> <closure-item-ref>
<function-obj> . <keyword> -> <closure-item-ref>
...

or <function-obj> [ <int> | <slice> | <keyword> ]

Using my '<<' grammer, positional assignment might be done like this:

fn << {<posn>:<value>, ...}

eg:
fn1 = fn << {1:'a'} # the key is an integer posn rather than an arg.
name.

...
<closure-item-ref> << <value>
Return a new function/closure with the selected fields from the
old one added or modified.

Maybe this is where __call__ comes in.
<closure-item-ref>(<value>, ...)

In addition to the currying operators one can add a 'curry' method that
behaves like the curry function proposed for the functional module.

That would mean, for a function...
pass
...you could write...
fn2 = fn [0] << "x" # Curry first parameter
fn2 = fn [-1] << "x" # Curry last parameter

fn2 = fn[0]("x")
fn2 = fn[-1]("x")

fn2 = fn.b << "x" # Curry keyword parameter

fn2 = fn["b"]("x")

fn2 = fn [1:2] << (1, 2) # Curry a slice

fn2 = fn[1:2](1, 2)

...
fn2.a.value () # Get value currently assigned to a

fn2["a"].value ()
fn2["a"] # Hmmm.
fn2.getargval("a") # ?

Having to use some method or property like 'value' is awkward. What other
ways are there? If an argument already has a value, this could be returned
instead of a <closure-item-ref>. Then a 'has_arg' method would be needed for
argument testing. This needs more thought. Anyways, implementing value
retrieval would require additional data be kept around if Python closures
are used to save the argument values. How important is it?

...
One advantage - it should be possible to write this as a library
without any language changes. I think.

from functional import curryfunction

def fn(<args>):
<body>
fn = curryfunction(fn)

At some point class 'function' will be rewritten to allow subclassing. At
that time curry operation methods could be added. At the very least the def
statement could be extended to provide a way to specify a function class
like the class statement and metaclasses. Then a subclass of 'function'
could define the curry operators.

Lenard Lindstrom
"<%s%s.%s>" % ("len-l.", 'telus', 'net')

Stephen Horne

2003-03-12 10:45:04 UTC

Permalink

On Wed, 12 Mar 2003 07:08:01 GMT, "Lenard Lindstrom"

Post by Lenard Lindstrom
"Stephen Horne" <intentionally at blank.co.uk> wrote in message

You may be onto something, but I'm not keen on your '|' version in
particular.

I see what you are saying, but to me the only one of your original
examples that seemed completely intuitive was '<<' - and that mainly
because I see it in the same sense as C++ stream insertion.

The '>>' looks like it should be removing something from the closure
(analogous with stream extraction).

The '|' version looks - well - I'm not sure, just not right.

Much of this is probably just the fact that you're re-using existing
operators, of course. My initial reaction to C++ stream insertion and
extraction operators was basically 'what - yuck!', but now I quite
like them.

Post by Lenard Lindstrom

Yes, I think the curried function object should be immutable. This leaves
out __setitem__. But is it needed?

The thought was...

<function-name> [<param-selection>] = <value>

...for currying, without needing to save a result or use alternative
operators.

Post by Lenard Lindstrom

How about...
<function-obj> [ <int> | <slice> ] -> <closure-item-ref>
<function-obj> . <keyword> -> <closure-item-ref>
...

or <function-obj> [ <int> | <slice> | <keyword> ]

OK - I was a bit confused there.

Post by Lenard Lindstrom
fn << {<posn>:<value>, ...}
fn1 = fn << {1:'a'} # the key is an integer posn rather than an arg.
name.

I think this creates a problem.

fn << 1 # Curries left parameter with rvalue
fn << { 2 : "x" } # Does not curry left parameter with rvalue -
# treats rvalues type as special case.

In short, I think this is a potentially confusing inconsistency.

Post by Lenard Lindstrom

<closure-item-ref> << <value>
Return a new function/closure with the selected fields from the
old one added or modified.

Maybe this is where __call__ comes in.
<closure-item-ref>(<value>, ...)

Not sure - call syntax suggests calling to strongly, which we are not
doing. I though it would be better to choose a syntax which emphasizes
that no call is done - only manipulation of closure contents and
selection flags.

Post by Lenard Lindstrom
In addition to the currying operators one can add a 'curry' method that
behaves like the curry function proposed for the functional module.

True

Post by Lenard Lindstrom

fn2.a.value () # Get value currently assigned to a

fn2["a"].value ()
fn2["a"] # Hmmm.
fn2.getargval("a") # ?
Having to use some method or property like 'value' is awkward.

True, but not IMO important.

Post by Lenard Lindstrom
Anyways, implementing value
retrieval would require additional data be kept around if Python closures
are used to save the argument values.

I didn't know that - I don't know how Python closures work, only what
the general principle behind closures is.

Post by Lenard Lindstrom
How important is it?

Not important - which is why an awkward syntax didn't seem to be a
problem.

BTW - 'fn2["a"]' without a method for getting the value was rejected
because it already has the closure-manipulating meaning. It's a shame,
but - as already noted - I didn't think it was that important.

Post by Lenard Lindstrom

One advantage - it should be possible to write this as a library
without any language changes. I think.

from functional import curryfunction
<body>
fn = curryfunction(fn)

Yes, but - erm - well - I don't like it!

I don't think you should need to convert a normal function into a
curryable function - functions should be functions, whether they
support currying or not.

Stephen Horne

2003-03-14 00:36:04 UTC

Permalink

On Thu, 13 Mar 2003 05:11:46 GMT, "Lenard Lindstrom"

Maybe my mistake shows that my
operator choices are confusing.

Overloading operators with completely new meanings (rather than with
the same meanings for new types) always causes some confusion - it's
almost a basic fact of life.

Actually, my biggest concern is with the
precedence of the '<<' operator.

OK - I understand.

I have a very strong tendency to avoid depending on precedence except
in cases where the precedence is absolutely clear. Therefore, if the
value to curry in is derived from an expression then that expression
should be in parentheses.

Meaning I wouldn't write...

fn1 = fn << x < 10 << x # Should this work?

I'd write...

fn1 = fn << (x < 10) << x

or rather, I'd write...

fn1 = fn [:] << ((x < 10), x)

To be honest, I'm not very keen on that syntax now either.

So this is a difference in mindset over what a call is. OK, I can see
that.

However, you are right in that what I suggest is not currying in the purest
sense.

Actually, I wasn't referring to that - I didn't even remember the
distinction until I read your explanation. Until you prodded my
memory, I'd forgotton that the result of currying all parameters
should be the fully evaluated result from the function. Its just that
to me, the closure-manipulation and final execution are different
things. Somehow, the idea that currying all parameters gives a
function requiring no parameters (but not the final result) seems more
natural. It's probably an artifact of my mostly imperative programming
experience.

The only time I used currying like that was in Miranda back at
college, about 10 years ago now. I used Haskell much more recently,
but didn't worry about currying because I was only getting the feel of
the language and didn't want to waste time learning things I *thought*
I already understood.

Now I think about that, I understand why you want a left-currying
syntax which fills the next available uncurried parameter - this was
what Miranda did most naturally because of its prefix-heavy syntax
with no parentheses for call parameters. IIRC a direct mult-parameter
function call such as '+ 1 1' could equally be seen as a sequence of
currying operations. So what I think as the general principle would,
to Miranda users at least, presumably be the fiddly exceptional case.

Actually allowing for argument value retrieval may not be such a big
problem. Keeping extra data around would only slow the currying operators.
The resulting functions would be just as fast. Anyway, there could always be
two curry classes, a bare-bones performance version as well as a full
featured version.

Hmmm - if it came to that, I'd start thinking that people who need
access to the curried-in values should keep their own records.

If a function returns a curried function, and there are two
curried-function classes, it needs to anticipate how that result will
be used. Assume the performance case and people can't access the
curried data. Assume the full-featured case and there are overheads
that cannot be easily removed - a conversion to the performance case
(discarding the extra data) would be possible but even that in itself
would waste some time.

Functional programming in Python" by David Mertz discusses currying,

I think I'll be doing some revision shortly.

Lenard Lindstrom

2003-03-13 05:11:46 UTC

Permalink

"Stephen Horne" <intentionally at blank.co.uk> wrote in message

Post by Stephen Horne
On Wed, 12 Mar 2003 07:08:01 GMT, "Lenard Lindstrom"
...

Post by Lenard Lindstrom
fn << {<posn>:<value>, ...}
fn1 = fn << {1:'a'} # the key is an integer posn rather than an arg.
name.

I think this creates a problem.
fn << 1 # Curries left parameter with rvalue
fn << { 2 : "x" } # Does not curry left parameter with rvalue -
# treats rvalues type as special case.

It does create a problem.

fn | {<posn>:<value, ...}

is consistant with my suggested syntax. Maybe my mistake shows that my
operator choices are confusing. Actually, my biggest concern is with the
precedence of the '<<' operator. An expression to the right the '<<' must be
parenthesized if in has bitwise or comparison operators. Yet in a function
call argument list only tuples need parentheses. This is confusing.

eg.

def fn(doprint, value): # Takes a boolean argument
if doprint: print 'value =', value
x = 5
fn(x < 10, x) # Works just fine
fn1 = fn << x < 10 << x # Should this work?
fn1() # fails because 'bool' object not
callable
fn1 = fn << (x < 10) << x # parentheses required.
fn1() # Now it works.

Unfortunately, every overloadable operation with precedence lower than '|'
has special behaviour.

Post by Stephen Horne
...

Post by Lenard Lindstrom
Maybe this is where __call__ comes in.
<closure-item-ref>(<value>, ...)

I intended the syntax to resemble a call. The idea behind currying is to
break down a function call into a series of single argument calls. I like
the __getitem__ approach with __call__ because now the implied calls are
obvious.

However, you are right in that what I suggest is not currying in the purest
sense.

for:
quad = lambda a, b, c, x: x(a*x + b) * c

result = quad << 1 << 2 << 3 << 2 # My syntax
or
result = quad[0:3] << (1, 2, 3, 2) # Your syntax
or
result = quad[0:3](1,2,3,2)

'result' is not 11 as it would be for currying, but a function object taking
no arguments. This allows for side effects. So which is preferable,
operators that emphasize the connection to actual currying or operators that
warn the programmer that this is something different? I am unsure on this.
My variation on 'currying' comes from PEP 309 and related postings.

Post by Stephen Horne
...

Post by Lenard Lindstrom
Anyways, implementing value
retrieval would require additional data be kept around if Python closures
are used to save the argument values.

I didn't know that - I don't know how Python closures work, only what
the general principle behind closures is.

Post by Lenard Lindstrom
How important is it?

Not important - which is why an awkward syntax didn't seem to be a
problem.

Post by Stephen Horne
...

Post by Lenard Lindstrom
from functional import curryfunction
<body>
fn = curryfunction(fn)

Yes, but - erm - well - I don't like it!
I don't think you should need to convert a normal function into a
curryable function - functions should be functions, whether they
support currying or not.

I don't like it either. This is the simplest way without requiring changes
to the language.

An aside:
For anyone interested, the three part online article "Charming Python:
Functional programming in Python" by David Mertz discusses currying,
closures, and other aspects of functional programming. It can be found at
the following URLs.

www-106.ibm.com/developerworks/library/l-prog.html
www-106.ibm.com/developerworks/library/l-prog2.html
www-106.ibm.com/developerworks/library/l-prog3.html

Lenard Lindstrom
"<%s@%s.%s>" % ("len-l.", "telus", "net")

Mike Meyer

2003-03-11 17:47:00 UTC

Permalink

How about...
<function-obj> . <keyword> -> <closure-item-ref>

def f(): return 1

...

f()

f.x = 3
print f.x

3

Why not use <function-obj> [ <string> ], or maybe <function-obj> {<keyword> }?

<mike

--
Mike Meyer <mwm at mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Stephen Horne

2003-03-11 12:41:27 UTC

Permalink

On Tue, 11 Mar 2003 03:50:30 GMT, "Lenard Lindstrom"

given that Python allows operator overloading, why not use some existing
operators for currying 'function' objects? The 'function' class has few
operations defined for it, so there remain many to choose from. Here are my

You may be onto something, but I'm not keen on your '|' version in
particular.

I would prefer defining the __len__, __getitem__, __setitem__ and
__delitem__ methods to allow direct manipulation (and creation and
deletion) of the closure. Though probably the function object should
be immutable.

How about...

<function-obj> [ <int> | <slice> ] -> <closure-item-ref>
<function-obj> . <keyword> -> <closure-item-ref>
Each returns a new type - a reference to the function with the
closure so far, plus an indication of the part of the closure
being referenced.

<closure-item-ref> ( args )
Call the function with the closure restricted to only the
referenced part.

<closure-item-ref> | <closure-item-ref>
Create a new function-and-closure, merging the items from both
(with priority given to the selected fields) and merging the
selection masks.

<closure-item-ref> << <value>
Return a new function/closure with the selected fields from the
old one added or modified.

That would mean, for a function...

def fn (a, b, c, d) :
pass

...you could write...

fn2 = fn [0] << "x" # Curry first parameter
fn2 = fn [-1] << "x" # Curry last parameter

fn2 = fn.b << "x" # Curry keyword parameter

fn2 = fn [1:2] << (1, 2) # Curry a slice

fn2 = fn [0] | fn [3] # Unselect (effectively uncurry) b and c

fn2.a.value () # Get value currently assigned to a

This is not properly thought through, though - and the '<<' probably
makes no sense to a lot of people.

One advantage - it should be possible to write this as a library
without any language changes. I think.

Jp Calderone

2003-03-11 18:32:47 UTC

Permalink

I would prefer defining the __len__, __getitem__, __setitem__ and
__delitem__ methods to allow direct manipulation (and creation and
deletion) of the closure.

... def __len__(self): return 0
... def __nonzero__(self): return 1
...

if foo(): print 'evaluates to true'

evaluates to true

Jp

--
"There is no reason for any individual to have a computer in their
home."
-- Ken Olson, President of DEC, World Future Society
Convention, 1977
--
up 8 days, 9:59, 8 users, load average: 0.00, 0.05, 0.07

Bernhard Herzog

2003-03-11 14:38:41 UTC

Permalink

I would prefer defining the __len__, __getitem__, __setitem__ and
__delitem__ methods to allow direct manipulation (and creation and
deletion) of the closure.

That would mean that the length of a function with no arguments is 0,
right? That's bad and will likely introduce subtle bugs because then all
functions with no arguments are suddenly considered false.

What about other callable objects? Classes, methods or instances that
have a __call__ method (They might even implement it through
__getattr__)?

Bernhard

--
Intevation GmbH http://intevation.de/
Sketch http://sketch.sourceforge.net/
MapIt! http://www.mapit.de/

Lenard Lindstrom

2003-03-11 03:50:30 UTC

Permalink

In PEP 309 a new operator, '@', was tentatively proposed for currying. But
given that Python allows operator overloading, why not use some existing
operators for currying 'function' objects? The 'function' class has few
operations defined for it, so there remain many to choose from. Here are my
suggestions:

operator '<<' for left currying. Usage:

<function object> << <argument value>

operator '|' for currying by key. Usage:

<function object> | (<argument key>, <argument value>) # a tuple
<function object> | [<argument key>, <argument value>] # a list
<function object> | {<key> : <value>, ...} # a dict

operator '>>' for right currying, if desired.

Each of the above operators would create a new function object. This may be
a wrapper function calling the curried function or a copy of the curried
function with altered closure and code objects.

The '[]' __getitem__ operator might also be defined for inspection of
argument / value bindings.

fn = lambda x, y, z: [x, y, z]
print (fn << 2 << 4)(6)

[2, 4, 6]

fn2 = fn | {'z':'z value', 'x':'x value'}
print (fn2 | ('y', 'y value'))()

['x value', 'y value', 'z value']

Adding new methods to the 'function' class should not require changes to its
other members or methods.

This test class wraps a function object to overload the '<<', '|', and '()'
operators:

class fnx(object):
"""fnx(fn) => callable object with overloaded operators for currying"""
def __init__(self, fn, **kwds):
if not callable(fn):
raise TypeError, "type not callable: '%s'" % type(fn).__name__
self._fn = fn
self._args = kwds.get('_args', ())
self._kwds = kwds.get('_kwds', {})
def __lshift__(self, other):
return fnx(self._fn, _args=self._args + (other,),
_kwds=dict(self._kwds))
def __or__(self, other):
if isinstance(other, dict):
kwds = dict(other)
elif isinstance(other, list) or isinstance(other, tuple):
kwds = dict((tuple(other),))
else:
raise TypeError, "unsupported operand type(s) for |: 'fnx' and
'%s'" % type(other).__name__
kwds.update(self._kwds)
return fnx(self._fn, _args=tuple(self._args), _kwds=kwds)
def __call__(self, *args, **kwds):
args = self._args + args
newkwds = dict(self._kwds)
newkwds.update(kwds)
return self._fn(*args, **newkwds)

Lenard Lindstrom
"<%s@%s.%s>" % ("len-l.", "telus", "net")

Marco Barisione

2003-03-13 10:47:43 UTC

Permalink

I would prefer using "%"; "function % something" should be equivalent
to"curry(function) % something". This would be similar to string
formatting.

class curry:
def __init__(self, fun, *args, **kwargs):
self.fun = fun
self.pending = args[:]
self.kwargs = kwargs.copy()
def __call__(self, *args, **kwargs):
if kwargs and self.kwargs:
kw = self.kwargs.copy()
kw.update(kwargs)
else:
kw = kwargs or self.kwargs
return self.fun(*(self.pending + args), **kw)
def __mod__(self, arg):
kw = self.kwargs.copy()
if isinstance(arg, (dict, UserDict.UserDict)):
kw.update(arg)
a = tuple(self.pending)
else:
a = tuple(self.pending) + tuple(arg)
return curry(self.fun, *a, **kw)

def foo(a, b, c): print a, b, c

call1 = foo % (1, 2, 3)
call1()

1 2 3

call2 = foo % {'a': 12, 'b': 123, 'c': 42}
call2()

12 123 42

call3 = foo % (1, 2)
call3(97)

1 2 97

call4 = foo % (1, 2) % {'c': 9}
call4()

1 2 9

BTW: Why?
Is currying so widespread?
I love Python because of its clear syntax, why should we make it less
clear?

I think the better solution is to add a "functional" module.

--
Marco Barisione
http://spazioinwind.libero.it/marcobari/

Bernhard Herzog

2003-03-12 12:13:18 UTC

Permalink

Post by Stephen Horne
Why would a function object be treated as a boolean?

Some functions accept optional callbacks. Naive implementations of such
functions might simply test whether the callback argument is true and
not whether it is None. I fell into a similar trap once when I added the
sequence interface to a class that didn't previously have it.

Of course, as Jp Calderone pointed out, this particular problem can be
worked around by also implementing __nonzero__ to return true even if
the length is zero. That would be a bit inconsistent with the other
builtin objects that support len, though.

Bernhard

--
Intevation GmbH http://intevation.de/
Sketch http://sketch.sourceforge.net/
MapIt! http://www.mapit.de/