Discussion:
Two GSOC Proposals: fonc compiler and autodiff
Robert Pfeiffer
2012-03-21 15:08:14 UTC
Permalink
Hello,

I am a student and would like to participate in the GSOC program. I would
like some feedback on these project ideas:

* Idea 1: A Clojure compiler based on fonc/cola
The CoLa Framework(http://piumarta.com/software/cola/) seems like a
promising basis for a native Clojure compiler. I have previously tinkered
with a simple implementation of Java based on CoLa(
http://www.hpi.uni-potsdam.de/hirschfeld/projects/delmdsoc/index.html). The
potion language, which is currently maintained by Mr. Fogus is also based
on some cola code.

** Brief Explanation:
Write a Clojure-compiler for cola. The compiler should generate native
code, but allow interactive/incremental development. Maybe write a C FFI.

** Expected Results:
-the persistent Clojure data structures in cola
-a Clojure reader in cola
-a compiler from Clojure data structures to cola

The cola compiler should pick up from there and do the native code
generation.

** Knowledge Prerequisite:
CoLa or alternatively C, Scheme and smalltalk. Parsing and compiler
construction.

** Skill Level:
Can somebody with cola experience comment on the feasibility of this? It
seems doable to me, but I do not know if the time is enough or whether
there are any mentors familiar with cola.


* Idea 2: Forward- and Reverse-mode autodiff
There are currently 3 ways to generate a program for function that computes
the derivative of another function:
-Symbolic: Take the source code of a program and generate the source code
of the program that computes the derivative.
-Numeric: Take a small epsilon and compute (f(x+epsilon)-f(x))/epsilon
-Automatic: Trace the execution of a program, replace the calculations with
calculations for the derivative.

Numeric differentiation is inexact, and Symbolic differentiation is tricky
because of unpredictable control flow.

** Brief Explanation:
An autodiff library for clojure would be very valuable for machine learning
and optimisation. At first we should implement forward-mode autodiff in
Clojure, then reverse-mode autodiff. The implementation should be
composable and avoid this bug:
http://eprints.nuim.ie/566/1/Perturbation.pdf. Maybe it should be
integrated with Incanter, so we can compute the derivatives of matrix
operations quickly.

As a proof-of-concept, learning in SVMs or neural networks would be nice,
because reverse mode autodiff is a generalisation of backprop.

** Expected Results:
A library with differentiation- and gradient- operators.

** Knowledge Prerequisite:
Some calculus.

** Skill level:
Easy to Medium.


Robert Pfeiffer
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
David Nolen
2012-03-21 15:59:16 UTC
Permalink
On Wed, Mar 21, 2012 at 11:08 AM, Robert Pfeiffer <
Post by Robert Pfeiffer
Hello,
I am a student and would like to participate in the GSOC program. I would
* Idea 1: A Clojure compiler based on fonc/cola
The CoLa Framework(http://piumarta.com/software/cola/) seems like a
promising basis for a native Clojure compiler. I have previously tinkered
with a simple implementation of Java based on CoLa(
http://www.hpi.uni-potsdam.de/hirschfeld/projects/delmdsoc/index.html).
The potion language, which is currently maintained by Mr. Fogus is also
based on some cola code.
This sounds neat though it seemed to me when following the latests STEPS
report that CoLa was superseded by Maru?
Post by Robert Pfeiffer
* Idea 2: Forward- and Reverse-mode autodiff
There are currently 3 ways to generate a program for function that
-Symbolic: Take the source code of a program and generate the source code
of the program that computes the derivative.
-Numeric: Take a small epsilon and compute (f(x+epsilon)-f(x))/epsilon
-Automatic: Trace the execution of a program, replace the calculations
with calculations for the derivative.
This sounds neat and a very interesting contribution to Incanter. I wonder
if David Liebke is up for mentoring :)

Would like for me to add these to the GSoC proposals page?

David
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Robert Pfeiffer
2012-03-21 16:37:06 UTC
Permalink
Post by David Nolen
This sounds neat though it seemed to me when following the latests STEPS
report that CoLa was superseded by Maru?
I have just looked at maru. It seems to be a metacircular Lisp compiler
without the PEG and the pepsi/smalltalk language. I need to read the new
report first. I didn't follow the fonc stuff since 2008; I had just read
the native compiler proposal and wanted to point out that
cola(pepi/id/fonc/maru) gives you a lot of infrastructure for a native
compiler for a dynamic language. Compiler construction is hard enough as it
is.

Would like for me to add these to the GSoC proposals page?
Yes please.

Robert Pfeiffer
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Timothy Baldridge
2012-03-21 17:08:21 UTC
Permalink
I had just read the
native compiler proposal and wanted to point out that
cola(pepi/id/fonc/maru) gives you a lot of infrastructure for a native
compiler for a dynamic language. Compiler construction is hard enough as it
is.
I'd just like to jump in here and mention that you might want to look
at PyPy's toolchain. PyPy has the ability to generate a
JIT/interpreter any arbitrary interpreter. And for-free you get FFI, a
JIT, a GC, etc. Since this is basically meta-programming, you can swap
out different GCs, backends (ARM, x86, x86-64, etc) all without ever
changing your sourcecode.

You'd end up writing stuff in RPython, but the resulting binaries are
going to way out-perform anything maru can produce. As a bonus (begin
shameless plug) it's possible to write RPython code in clojure via
clojure-py.

An excellent overview of RPython is available here:

http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time
http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-the-speed-of-pypy-83-3279075

Timothy
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
David Nolen
2012-03-21 18:02:18 UTC
Permalink
While we're on the subject of other approaches - adapting the ClojureScript
compiler for LLVM and clojure-scheme are two other valid paths towards
native compilation.

David
Post by Timothy Baldridge
I had just read the
native compiler proposal and wanted to point out that
cola(pepi/id/fonc/maru) gives you a lot of infrastructure for a native
compiler for a dynamic language. Compiler construction is hard enough as
it
is.
I'd just like to jump in here and mention that you might want to look
at PyPy's toolchain. PyPy has the ability to generate a
JIT/interpreter any arbitrary interpreter. And for-free you get FFI, a
JIT, a GC, etc. Since this is basically meta-programming, you can swap
out different GCs, backends (ARM, x86, x86-64, etc) all without ever
changing your sourcecode.
You'd end up writing stuff in RPython, but the resulting binaries are
going to way out-perform anything maru can produce. As a bonus (begin
shameless plug) it's possible to write RPython code in clojure via
clojure-py.
http://tratt.net/laurie/tech_articles/articles/fast_enough_vms_in_fast_enough_time
http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-the-speed-of-pypy-83-3279075
Timothy
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
David Nolen
2012-03-21 23:22:47 UTC
Permalink
On Wed, Mar 21, 2012 at 12:37 PM, Robert Pfeiffer <
Post by Robert Pfeiffer
Post by David Nolen
This sounds neat though it seemed to me when following the latests STEPS
report that CoLa was superseded by Maru?
I have just looked at maru. It seems to be a metacircular Lisp compiler
without the PEG and the pepsi/smalltalk language. I need to read the new
report first. I didn't follow the fonc stuff since 2008; I had just read
the native compiler proposal and wanted to point out that
cola(pepi/id/fonc/maru) gives you a lot of infrastructure for a native
compiler for a dynamic language. Compiler construction is hard enough as it
is.
Would like for me to add these to the GSoC proposals page?
Yes please.
Done!
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
kovas boguta
2012-03-21 23:32:53 UTC
Permalink
Hadn't seen that project before, pretty cool.

My 2 cents, in terms of clojure compiler-related stuff:

Reimplementing the persistent datastructures in terms of clojure
protocols seems like a high-leverage thing to do, and the basis of a
lot of things.

Before this happens, you can't have a real clojure-in-clojure
implementation. Once you have it, going all the way seems more
feasible.

Independent of the clojure-in-clojure implementation, it would also
give immediate benefit to other current and future clojure
implementations such as clojurescript.

Clojure protocols were basically designed with this use case in mind,
so it would be pretty cool to have the idea taken to fruition.


On Wed, Mar 21, 2012 at 11:08 AM, Robert Pfeiffer
Post by Robert Pfeiffer
Hello,
I am a student and would like to participate in the GSOC program. I would
* Idea 1: A Clojure compiler based on fonc/cola
The CoLa Framework(http://piumarta.com/software/cola/) seems like a
promising basis for a native Clojure compiler. I have previously tinkered
with a simple implementation of Java based on
CoLa(http://www.hpi.uni-potsdam.de/hirschfeld/projects/delmdsoc/index.html).
The potion language, which is currently maintained by Mr. Fogus is also
based on some cola code.
Write a Clojure-compiler for cola. The compiler should generate native code,
but allow interactive/incremental development. Maybe write a C FFI.
-the persistent Clojure data structures in cola
-a Clojure reader in cola
-a compiler from Clojure data structures to cola
The cola compiler should pick up from there and do the native code
generation.
CoLa or alternatively C, Scheme and smalltalk. Parsing and compiler
construction.
Can somebody with cola experience comment on the feasibility of this? It
seems doable to me, but I do not know if the time is enough or whether there
are any mentors familiar with cola.
* Idea 2: Forward- and Reverse-mode autodiff
There are currently 3 ways to generate a program for function that computes
-Symbolic: Take the source code of a program and generate the source code of
the program that computes the derivative.
-Numeric: Take a small epsilon and compute (f(x+epsilon)-f(x))/epsilon
-Automatic: Trace the execution of a program, replace the calculations with
calculations for the derivative.
Numeric differentiation is inexact, and Symbolic differentiation is tricky
because of unpredictable control flow.
An autodiff library for clojure would be very valuable for machine learning
and optimisation. At first we should implement forward-mode autodiff in
Clojure, then reverse-mode autodiff. The implementation should be composable
and avoid this bug: http://eprints.nuim.ie/566/1/Perturbation.pdf. Maybe it
should be integrated with Incanter, so we can compute the derivatives of
matrix operations quickly.
As a proof-of-concept, learning in SVMs or neural networks would be nice,
because reverse mode autodiff is a generalisation of backprop.
A library with differentiation- and gradient- operators.
Some calculus.
Easy to Medium.
Robert Pfeiffer
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to ***@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
Loading...