Macroexpansion of with-html-output body?

[Eli Naeher]

Hello,

It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list. In particular, it would allow the user to
create a tag which modifies its body (as a tree) even when that body
contains the expansion of another user-defined tag. I don't believe
this is currently possible, as this hypothetical outer tag would have
access only to the HTML strings returned by the
convert-tag-to-string-list specialized on the inner tag and not to the
relevant s-expressions.

Is there any interest in a patch implementing this behavior?

Thanks,

--Eli

[Jeff Cunningham]

Post by Eli Naeher
Hello,
It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list. In particular, it would allow the user to
create a tag which modifies its body (as a tree) even when that body
contains the expansion of another user-defined tag. I don't believe
this is currently possible, as this hypothetical outer tag would have
access only to the HTML strings returned by the
convert-tag-to-string-list specialized on the inner tag and not to the
relevant s-expressions.
Is there any interest in a patch implementing this behavior?
Thanks,
--Eli
____________

I'm not sure I follow what you are suggesting. Could you give a simple
example?

Thanks
--Jeff

[Eli Naeher]

Post by Jeff Cunningham
I'm not sure I follow what you are suggesting. Could you give a simple
example?

Sure -- my explanation was not a model of clarity. Here's what I've
been playing with:

(cl-who:with-html-output (var test-file :prologue t :indent t)
(with-numbered-figures ("Fig. ~@R")
(:h2 "Here are some images.")
(:p "It is interesting to examine foo and bar over the past"
(:em "n") "years:")
(png-inline ("Foo vs. Bar Over Time" stream)
(adw-charting:with-line-chart (400 100)
(adw-charting:add-series "Foo" '((1 1) (2 1) (3 5) (4 8)))
(adw-charting:add-series "Bar" '((1 -3) (4 -2) (6 -1) (8 7)))
(adw-charting:save-stream stream)))
(:p "Here is a picture of my cat:")
(:img :src "picture-of-my-cat.jpg" :alt "My Cat.")))

png-inline is a macro that lets its body write binary PNG data to
stream and expands to:

(:img :src "<long RFC 2397 data: URI with base64-encoded PNG>"
:alt "Foo vs. Bar Over Time")

with-numbered-figures finds all the lists like (:img ...) in its body
and inserts:

(:div :class "caption" "Figure n")

after each one, where "Figure n" is an iterator formatted with the
specified format string.

The functionality of either of these on their own could be achieved
simply by writing macros which themselves use with-html-output and
expand to strings. However, in order for with-numbered-figures to
correctly find the (:img ...) created by the expansion of png-inline,
it is necessary that the latter expand to w-h-o forms, and, if we want
to be able to also use png-inline in w-h-o directly without using
with-numbered-figures, it is necessary that w-h-o perform
macroexpansion during its tree-walking, or else png-inline will be
expanded subsequent to the w-h-o expansion and Lisp will see it as an
attempt to call an :img function.

I hope this is a little clearer (and that GMail does not break the
formatting too badly).

Thanks,

--Eli

[Jeff Cunningham]

Post by Eli Naeher
png-inline is a macro that lets its body write binary PNG data to
(:img :src "<long RFC 2397 data: URI with base64-encoded PNG>"
:alt "Foo vs. Bar Over Time")
with-numbered-figures finds all the lists like (:img ...) in its body

I am curious as to how you have implemented png-inline. Would you mind
sharing it?

Regards.
--Jeff

[Volkan YAZICI]

Post by Eli Naeher
It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list.

IIRC, there had been some similar discussions on the list about this
matter. Expanding found macros during walking the tree can be quite
useful. (OTOH, this method may have some problems, that I cannot find
one now.)

Post by Eli Naeher
In particular, it would allow the user to create a tag which
modifies its body (as a tree) even when that body contains the
expansion of another user-defined tag. I don't believe this is
currently possible, as this hypothetical outer tag would have access
only to the HTML strings returned by the convert-tag-to-string-list
specialized on the inner tag and not to the relevant s-expressions.
Is there any interest in a patch implementing this behavior?

I hope so. At least, I do. If you have one ready (or you're
volunteered to implement one) let's see it in action.


Regards.

[Edi Weitz]

Post by Eli Naeher
It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list. In particular, it would allow the user
to create a tag which modifies its body (as a tree) even when that
body contains the expansion of another user-defined tag. I don't
believe this is currently possible, as this hypothetical outer tag
would have access only to the HTML strings returned by the
convert-tag-to-string-list specialized on the inner tag and not to
the relevant s-expressions.
Is there any interest in a patch implementing this behavior?

Modifying CL-WHO's internals to allow this is on my todo list, but
there are a lot more things on this list, so the chances of me doing
this myself in the near future are not very big. So, if someone wants
to do this, I'm all for it. Please read this first, though:

http://weitz.de/patches.html

In the case of CL-WHO I'd think that backwards compatibility would be
pretty important.

Edi.

[Jeff Cunningham]

Post by Edi Weitz

Post by Eli Naeher
It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list. In particular, it would allow the user
to create a tag which modifies its body (as a tree) even when that
body contains the expansion of another user-defined tag. I don't
believe this is currently possible, as this hypothetical outer tag
would have access only to the HTML strings returned by the
convert-tag-to-string-list specialized on the inner tag and not to
the relevant s-expressions.
Is there any interest in a patch implementing this behavior?

Modifying CL-WHO's internals to allow this is on my todo list, but
there are a lot more things on this list, so the chances of me doing
this myself in the near future are not very big. So, if someone wants
http://weitz.de/patches.html
In the case of CL-WHO I'd think that backwards compatibility would be
pretty important.
Edi.

I also like the idea.
And as I have tens of thousands of lines of cl-who code in operation, I
emphatically vote for backwards compatibility.

--Jeff

[Victor Kryukov]

Post by Jeff Cunningham

Post by Edi Weitz

Post by Eli Naeher
It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list. In particular, it would allow the user
to create a tag which modifies its body (as a tree) even when that
body contains the expansion of another user-defined tag. I don't
believe this is currently possible, as this hypothetical outer tag
would have access only to the HTML strings returned by the
convert-tag-to-string-list specialized on the inner tag and not to
the relevant s-expressions.

Modifying CL-WHO's internals to allow this is on my todo list, but
there are a lot more things on this list, so the chances of me doing
this myself in the near future are not very big. So, if someone wants
http://weitz.de/patches.html
In the case of CL-WHO I'd think that backwards compatibility would be
pretty important.
Edi.

I also like the idea.
And as I have tens of thousands of lines of cl-who code in operation,
I emphatically vote for backwards compatibility.

Below is my attempt to implement macroexpansion capability in
with-html-output body. Before sending it formally as a patch I want to
get your feedback on the implementation and also ask for some testing.

The idea is simple: we define new special variable *MACRO-TO-EXPAND*
which stores names of all the macros that should be expanded before
with-html-output comes into play. Notice that specail forms 'htm',
'esc', 'str' and 'fmt' are no exception here, and may be defined as
similar macros. That simplifies code of TREE-TO-TEMPLATE,
TREE-TO-COMMANDS-AUX and TREE-TO-COMMANDS significantly.

Below[1] is an example of how we're using this functionality.
def-syntax-macro is a syntactic sugar that defines a normal macro and
also adds its name to *MACRO-TO-EXPAND*

(defpackage :cl-who-example
(:use :cl :cl-who))

(in-package :cl-who-example)

(cl-who::def-syntax-macro html-list (&body body)
`(:ul
,@(loop
for elem in body
collect `(:li ,elem))))

(with-html-output-to-string (s)
(:html
(:title "Title"))
(:body
(:h1 "H1 header")
(:p "A small list"
(html-list
"First"
"Second"
(:a :href "/link" "Third")))))

=> "<html><title>Title</title></html><body><h1>H1 header</h1><p>A small list<ul><li>First</li><li>Second</li><li><a href='/link'>Third</a></li></ul></p></body>"

To test new functionality, simply load the code in who1.lisp[2] after you
load CL-WHO - that will allow you to keep the original CL-WHO intact.

I'm also attaching very simple test to check that we haven't braken
things at least for the three examples advertised on CL-WHO web page.

Please let me know your feedback,

Victor.

[Victor Kryukov]

Post by Victor Kryukov

Post by Jeff Cunningham

Post by Edi Weitz

Post by Eli Naeher
It seems like it would be nice to expand any macros detected while
walking the tree of the w-h-o body. This would allow users to define
their own pseudo-tags a little more flexibly than is possible with
convert-tag-to-string-list. In particular, it would allow the user
to create a tag which modifies its body (as a tree) even when that
body contains the expansion of another user-defined tag. I don't
believe this is currently possible, as this hypothetical outer tag
would have access only to the HTML strings returned by the
convert-tag-to-string-list specialized on the inner tag and not to
the relevant s-expressions.

Modifying CL-WHO's internals to allow this is on my todo list, but
there are a lot more things on this list, so the chances of me doing
this myself in the near future are not very big. So, if someone wants
http://weitz.de/patches.html
In the case of CL-WHO I'd think that backwards compatibility would be
pretty important.
Edi.

I also like the idea.
And as I have tens of thousands of lines of cl-who code in operation,
I emphatically vote for backwards compatibility.

Below is my attempt to implement macroexpansion capability in
with-html-output body. Before sending it formally as a patch I want to
get your feedback on the implementation and also ask for some testing.

It also seems to resolve (with-html-output-to-str (str) ...) bug.

Regards,
Victor.

[Jeff Cunningham]

Post by Victor Kryukov
To test new functionality, simply load the code in who1.lisp[2] after you
load CL-WHO - that will allow you to keep the original CL-WHO intact.
I'm also attaching very simple test to check that we haven't braken
things at least for the three examples advertised on CL-WHO web page.
Please let me know your feedback,
Victor.

Victor,

So far so good. Your mod is running now on one live test site and
there's no sign of breakage. I'll do more extensive testing tomorrow.

Jeff

[Victor Kryukov]

Post by Jeff Cunningham
Victor,
So far so good. Your mod is running now on one live test site and
there's no sign of breakage. I'll do more extensive testing tomorrow.

Jeff,

That was fast! I'm holding my breath...

Victor.

[Jeff Cunningham]

Post by Victor Kryukov
Jeff,
That was fast! I'm holding my breath...
Victor.

Victor,

I'm not sure anything I'm doing really exercises your changes, but I now
have it running on three servers without any sign of trouble (or
difference in output, for that matter).

I sat down this evening to try to understand it better, and for awhile I
thought I might be able to use it to solve a macro expansion problem
that's been bothering me. I don't think it can be applied, but I'm not
sure. Here's a simplified version of a macro I use :

(defmacro section (title &rest body)
`(with-html-output (*standard-output* nil :indent t)
(:table
(:a :name ,title)
(:tr
(:td
(:h2 (str ,title))
,@body)))))

Here is a trivial example showing how I use it:

(section "A Title"
(:p "A cl-who formatted form")
(:p "The body could be any collection of cl-who code"))

What I'd like to be able to do is feed it unexpanded cl-who code like this:

(let ((code1 '("A Title"
(:p "A cl-who formatted form")
(:p "The body could be any collection of cl-who code")))
(code2 '("Another title"
(:h2 "Different code here"))))
(section* code1)
(section* code2))

And I've tried to make the first macro work by wrapping it in a second
one like this:

(defmacro section* (lst)
(let ((ttl (gensym))
(bdy (gensym)))
`(let ((,ttl (car ,lst))
(,bdy (cdr ,lst)))
(section ,ttl ,bdy))))

Of course, the last form isn't right and evaluates without the body
statements. I had expected to use ,@bdy in the last form, but that give
an error.

Do you have any idea how to do something like this?

--Jeff

Post by Victor Kryukov
_______________________________________________
cl-who-devel site list
http://common-lisp.net/mailman/listinfo/cl-who-devel

[Victor Kryukov]

Post by Jeff Cunningham
Victor,
I'm not sure anything I'm doing really exercises your changes, but I
now have it running on three servers without any sign of trouble (or
difference in output, for that matter).

Well, if you code ever uses special symbols str, esc, fmt or htm -
that's a good sign - it means that macroexpansion is working.

Post by Jeff Cunningham
I sat down this evening to try to understand it better, and for awhile
I thought I might be able to use it to solve a macro expansion problem
that's been bothering me. I don't think it can be applied, but I'm not

I highly recommend readign cl-who code. It's an easy read - the code
is short, very well written and has just the right amount of comments.

Post by Jeff Cunningham
(defmacro section (title &rest body)
`(with-html-output (*standard-output* nil :indent t)
(:table
(:a :name ,title)
(:tr
(:td
(:h2 (str ,title))

With the new macro def-syntax-macro, you can achieve the same with the
following code:

(cl-who::def-syntax-macro section (title &rest body)
`(:table
(:a :name ,title)
(:tr (:td (:h2 (str ,title)) ,@body))))

BTW - your (:a :name ,title) inside a table looks really suspicious!

Now whenever you use section macro inside your with-html-output, it
will expanded _before_ with-html-output comes into play. Note
cl-who:: qualifier before the def-syntax-macro name - I haven't
touched original cl-who files for the purpose - it's much easier to
try the changes this way - and therefore def-syntax-macro is not
exported by default, hence the cl-who:: package specification.

Post by Jeff Cunningham
(section "A Title"
(:p "A cl-who formatted form")
(:p "The body could be any collection of cl-who code"))

If you define section with the def-syntax-macro as above, you can
legally use this inside your with-html-output macro.

Post by Jeff Cunningham
(let ((code1 '("A Title"
(:p "A cl-who formatted form")
(:p "The body could be any collection of cl-who code")))
(code2 '("Another title"
(:h2 "Different code here"))))
(section* code1)
(section* code2))

Well - are you _sure_ you really need that? Currently, cl-who doesn't
provide any interface for producing code out of sexps, and if you read
who.lisp or cl-who documentation - esp. paying attention to "Syntax
and Semantics" section - you'll understand why.

In most cases, you just don't need that. If you _really_ insist, try
this one:

(defpackage :cl-who-sexp
(:use :cl :cl-who))

(in-package :cl-who-sexp)

(defun sexp-with-html-output (sexp)
"Generates string out of SEXP according to CL-WHO rules"
(with-output-to-string (s)
(eval (cl-who::tree-to-commands sexp s))))

CL-WHO-SEXP> (sexp-with-html-output '(:html (:title "title")))
"<html></html><title>title</title>"

This is either a very clever or very ugly hack. It would be better to
re-write tree-to-commands-aux to provide such functionality instead.

Regards,

Victor
--
http://macrodefinition.blogspot.com/

[Jeff Cunningham]

Wow - talk about fast! Victor, I'm impressed.

Post by Victor Kryukov
Well, if you code ever uses special symbols str, esc, fmt or htm -
that's a good sign - it means that macroexpansion is working.

I use all of them, so it's working.

Post by Victor Kryukov

Post by Jeff Cunningham
I sat down this evening to try to understand it better, and for awhile
I thought I might be able to use it to solve a macro expansion problem
that's been bothering me. I don't think it can be applied, but I'm not

I highly recommend readign cl-who code. It's an easy read - the code
is short, very well written and has just the right amount of comments.

Okay, I'll do that.

Post by Victor Kryukov
With the new macro def-syntax-macro, you can achieve the same with the
(cl-who::def-syntax-macro section (title &rest body)
`(:table
(:a :name ,title)
BTW - your (:a :name ,title) inside a table looks really suspicious!

It should - it is a remnant of my simplification (belonged to a
preceding row). I was trying to hack out everything not pertinent to the
example.

Post by Victor Kryukov
Now whenever you use section macro inside your with-html-output, it
will expanded _before_ with-html-output comes into play. Note
cl-who:: qualifier before the def-syntax-macro name - I haven't
touched original cl-who files for the purpose - it's much easier to
try the changes this way - and therefore def-syntax-macro is not
exported by default, hence the cl-who:: package specification.

Post by Jeff Cunningham
(section "A Title"
(:p "A cl-who formatted form")
(:p "The body could be any collection of cl-who code"))

If you define section with the def-syntax-macro as above, you can
legally use this inside your with-html-output macro.

Excellent.

Post by Victor Kryukov

Post by Jeff Cunningham
(let ((code1 '("A Title"
(:p "A cl-who formatted form")
(:p "The body could be any collection of cl-who code")))
(code2 '("Another title"
(:h2 "Different code here"))))
(section* code1)
(section* code2))

Well - are you _sure_ you really need that? Currently, cl-who doesn't
provide any interface for producing code out of sexps, and if you read
who.lisp or cl-who documentation - esp. paying attention to "Syntax
and Semantics" section - you'll understand why.

I'm not at all sure, and you're probably right that its a bad idea.
Still, I'd like to understand it.
Interesting... I just tried to write down what led me to it and in the
process thought of a much simpler way.

Post by Victor Kryukov
In most cases, you just don't need that. If you _really_ insist, try
(defpackage :cl-who-sexp
(:use :cl :cl-who))
(in-package :cl-who-sexp)
(defun sexp-with-html-output (sexp)
"Generates string out of SEXP according to CL-WHO rules"
(with-output-to-string (s)
(eval (cl-who::tree-to-commands sexp s))))
CL-WHO-SEXP> (sexp-with-html-output '(:html (:title "title")))
"<html></html><title>title</title>"
This is either a very clever or very ugly hack. It would be better to
re-write tree-to-commands-aux to provide such functionality instead.

Its a fascinating piece of code. I'm going to play with it. Thanks!

Regards,

Jeff

[Maciej Pasternacki]

Post by Victor Kryukov
Below is my attempt to implement macroexpansion capability in
with-html-output body. Before sending it formally as a patch I want to
get your feedback on the implementation and also ask for some testing.

Works for me. No CL-WHO code base on my side, but macroexpanding HTML
makes enables me to use CL-WHO instead of YACLML (and, actually, allowed
me to write a drop-in replacement for using YACLML syntax with a "<"
package -- I like it more than CL-WHO's keywords since with (<:a :href
"...") tags stand out visually).

+1 for including this or similar functionality in CL-WHO. This probably
would make me switch -- without it, YACLML is still more programmable
than CL-WHO.

Regards,
Maciej.

[Daniel Gackle]

< This probably would make me switch >

Me too. I started on CL-WHO but switched to LML2 for the same reason.

Dan

Post by Maciej Pasternacki

Post by Victor Kryukov
Below is my attempt to implement macroexpansion capability in
with-html-output body. Before sending it formally as a patch I want to
get your feedback on the implementation and also ask for some testing.

Works for me. No CL-WHO code base on my side, but macroexpanding HTML
makes enables me to use CL-WHO instead of YACLML (and, actually, allowed
me to write a drop-in replacement for using YACLML syntax with a "<"
package -- I like it more than CL-WHO's keywords since with (<:a :href
"...") tags stand out visually).
+1 for including this or similar functionality in CL-WHO. This probably
would make me switch -- without it, YACLML is still more programmable
than CL-WHO.
Regards,
Maciej.
--
-><- Maciej 'japhy' Pasternacki -><- http://www.pasternacki.net/ -><-
_______________________________________________
cl-who-devel site list
http://common-lisp.net/mailman/listinfo/cl-who-devel

[Victor Kryukov]

Post by Victor Kryukov
Below is my attempt to implement macroexpansion capability in
with-html-output body. Before sending it formally as a patch I want to
get your feedback on the implementation and also ask for some testing.

I created a public git repository to make it easier to try
"macroexpand" extension. You can get it with git at

git://tunespresso.com/cl-who.git

Original, untouched cl-who version 0.11 is in master branch, and my
extension is in macroexpand branch.

To use it, install git and execute the following commands:

git clone git://tunespresso.com/cl-who.git (1)
git checkout origin/macroexpand (2)

(1) clones git repository into cl-who subdirectory of the current
directory
(2) checks out macroexpand branch

To switch back to the original cl-who, use the following command:

git checkout master

Branch switching is done in place, so you don't have to modify your
.asd links, etc.

Best Regards,
Victor.

[Victor Kryukov]

Post by Victor Kryukov

Post by Victor Kryukov
Below is my attempt to implement macroexpansion capability in
with-html-output body. Before sending it formally as a patch I want to
get your feedback on the implementation and also ask for some testing.

I created a public git repository to make it easier to try
"macroexpand" extension. You can get it with git at
git://tunespresso.com/cl-who.git
Original, untouched cl-who version 0.11 is in master branch, and my
extension is in macroexpand branch.
git clone git://tunespresso.com/cl-who.git (1)
git checkout origin/macroexpand (2)
(1) clones git repository into cl-who subdirectory of the current
directory

(1.5) cd cl-who/

Post by Victor Kryukov
(2) checks out macroexpand branch

while in cl-who directory tracked by git

Post by Victor Kryukov
git checkout master
Branch switching is done in place, so you don't have to modify your
.asd links, etc.

Regards,
Victor.