root/project_planning/TALK

The Pugs team is porting all content in this directory to the Pugs Trac
at http://dev.pugscode.org. Please put new content there ;-)

----

Feel free to use wikipedia Talk style if you know it.

== Purpose ==

This directory was created to address these problems:
 - No one knows what the current state of the project is.
 - It is difficult to determine what people are working on.
 - No one knows what needs to be done.
 - No one knows how they can help.
 - No one knows where we are going, or how we are to get there.
Perhaps this is a slight overstatement.  But only a slight one.

The project management approach of pugs has been anarchistic
development.  A flooding algorithm for planning (everyone working on
whatever development path/aspects appeals to them); active and rapid
development; continuous communication; and a low commit threshold
("don't worry about breaking the build, just commit, and we'll fix
it").  This approach worked well when development was rapid, there
was a clear primary development thrust, and an active community with
an up to date and detailed organizational memory.

For several reasons (we haven't been able to pay audreyt to work full
time being a key one), the main thrust has slowed, the overall pace of
development has slowed, and the community has become smaller, less
active, and less tightly informed.  This created negative feedback.
It became harder to tell what to work on, easier to get stuck, and
harder to be motivated.  Thus further slowing development and
shrinking the active community.

I created this directory to increase project transparency.  To make it
easier for people to see what needs doing, and why.  This is not a
full solution, but will hopefully mitigate the negative feedback loop,
helping us make less slow progress until things pick up again.

-- putter 2007-01-20


== Fix PIL2JS ==

PIL2JS is currently broken.
It is our second most complete implementation of p6, after pugs.

$ ./pugs -Bjs -e 'say 3;'
$
No output.
"make test-js" fails similarly.

<audreyt> the problem is that JS.pm needs to accomodate statement:cond form
  which unlike the :if form, takes two arguments as full closures instead of blocks
  and so will need to do the pil2js equiv of closure apply .()
  I'm checking in something that doesn't do the .() -- it's 7am here [...]

See r15128.
perl5/PIL2JS/lib/Prelude/JS.pm
  register "&statement_control:cond" => sub {...

-- putter 2007-01-22


== KP6 on PIL2JS? ==

Could kp6 run on PIL2JS?  If so, then kp6-pil2js would become our
second self-hosting p6 implementation.  pil2js has good smoke results,
and so might be a faster way to get kp6 "real" than writing a new
backend from scratch.

pil2js currently delegates its parsing to pugs, and so can't parse oo,
but does use oo internally.  so porting kp6 might require hand
transfering the oo parts to get bootstrapped.

Some things which might be explored:
 - how much of kp6 can pil2js run now?
 - can kp6 handle pil2js's p6 code?  (perl5/PIL2JS/lib6/Prelude)
 - how close is the kp6 ast to the PIL1 ast pil2js uses?

-- putter 2007-01-22


== redsix on PIL2JS ==
-- putter 2007-01-24

Background

PIL2JS uses pugs to generate PIL1 ast, processes it using p5, and has
a p6+js runtime.  It has been blocked for two years by PIL1 not having
oo information.

PIL2JS is attractive.  It was our second best (after pugs) backend in
terms of test coverage.  It's use of p6 and p5 means it is closer than
pugs to being a p6 bootstrap.  Even some of its js could be rewritten
in p6, if oo parsing was available.  Its performance, while slower
than pugs, seemed tolerable.  jsperl5 is a variant of pil2js which
allows it to use p5.

Redsix, written in ruby, has a p6 parser.  The grammar was derived
from pugs's.  While buggy, it aspires to completeness.  It uses
p5-syntax regexps plus rules.  It has a working operator precedence
parser, though it's non-spec.  Performance is good - it was running
faster than pugs at one point.

An implementation of p5-syntax regexps plus rules exists for p5.
Performance is good - it may be faster than the ruby one.

Proposal

Create an alternate p6 parser for pil2js.  Translate the redsix
grammar and opp to p5.  Write a new emitter for it, based on pil2js's
PIL1 emitter.  Get pil2js and the new parser working together.

Tweak the new parser until it can handle the pil2js p6 code.
At that point, pil2js is free to evolve independently of pugs.

Development can then proceed in several directions.
Improve the new parser to recover test regressions from the switch.
Add emitting for oo declarations.
Then:
Migrate the p5 bits to p6.
At that point, we have a bootstrapped p6-on-p6, for a large subset of p6.

Disadds/Risks

The emitter backend is new.  Redsix was passing less than 20% of
tests, so the ability of the grammar to correctly handle the other 80%
is uncertain/unlikely without more work.

Compared with running kp6 on pil2js, this is a harder and more complex
approach.  Though one with the potential to more quickly result in a
relatively complete p6 implementation.

pugs can *already* do parsing and oo.  Faster and better than the
result of this exercise will be able to until after quite a bit of
work.  Even then, if you have to move to another, faster backend, you
might as well have developed on pugs instead, and emitted to the new
backend from there.

An alternative is to migrate pil2js into a new pil2p6.  Run it on
pugs.  Then it can be developed in a full p6 dialect, rather than
playing bootstrap games.  The redsix parser can be migrated to p6,
full p6, if that still seems the right way to develop a parser.
Bootstrap happens when it is ready to happen.  The main risk of this
approach is the possibility of pugsbugs interfering with development.

So the key assumptions of this proposal are that (1) pugs won't be
able to output an oo ast for pil2js anytime soon; and (2) pugs is
currently too buggy to develop a pil2p6 on it.  It's not entirely
clear either of these is true.  If (2) is untrue, a better path to
p6-on-p6 is pil2p6.  If (1) is untrue, a better path is to wait,
perhaps filling time by migrating the redsix parser to p6, or fleshing
out kp6.
Counter: (1) may turn out to be untrue, but it is certainly a
plausible hypothesis.  And even if untrue, the only impact is it would
be better to migraite the redsix parser to p6 instead of p5.  Which
doesn't seem a deadly issue.  Regards (2), unlike creating an
alternative parser for pil2js, pil2p6 is setting aside working code,
and hoping a non-trivial translation to p6 will sort of work too --
the proposal has more of a "keep working code working" flavor than pil2p6.


== Integrating KP6 with the test suite ==

It would be nice to run the test suite against kp6.
It is easy, and will help make it obvious what is and is not working.
:Counter argument: it will only show nothing is working, and is thus
 uninteresting.  I expect only 01-sanity/01-tap.t will pass, as
 variable assignment doesn't work yet.  -- putter 2007-01-22
::But keeping an eye on that, and motivating improvement, is exactly
  what the testing is for.  *Everything* which is a p6 implementation
  should be able to run the test suite, no?  -- putter 2007-01-22

* Create a friendlier "program api" to kp6.
  It currently spits p5 on stdout.
  * Create a run-kp6.pl analogous to mp6's run-mp6.pl.
* Make ./pugs -Bkp6 call run-kp6.pl.
  src/Pugs.hs.  Perhaps follow the js code there.
  Hopefully only one or two lines will be needed.
* Add a test-kp6 target in Makefile.PL.
* Make smoke work.
  * Add a smoke-kp6 target in Makefile.PL.
  * Tell util/run-smoke.pl about kp6.
    Just a $ENV{HARNESS_PERL} modification like the other ones there.

-- putter 2007-01-21