Code generation, Poco Script, and Servlets

[klondenberg]

Hello,

I'm new in this community, so just as an introduction, as you might see from the following, I have already done a bit of research into this topic before I noticed Poco, so you might consider it valuable what I found and planned.

I opened a new topic since this one touches on both Servlets and Poco Script, so neither topic seemed appropriate, but you might consider this as a continuation of both threads.

First of all, don't dismiss the idea of pre-compiled pages much too early. This is a very powerful concept, JSP just did it the wrong way by introducing low level code into the jsps.

But apart from that, scripting capabilities are often a neccessity. I see pre-compiled pages as a complementary, not as a contradicting technology to generated code.

Look here: http://www.swig.org/ - this is pretty much what you were looking for. I have not tried it, this is an excerpt from their website:

---
SWIG is a software development tool that connects programs written in C and C++ with a variety of high-level programming languages. SWIG is used with different types of languages including common scripting languages such as Perl, PHP, Python, Tcl, Ruby and PHP. The list of supported languages also includes non-scripting languages such as C#, Common Lisp (CLISP, Allegro CL, CFFI, UFFI), Java, Modula-3 and OCAML. Also several interpreted and compiled Scheme implementations (Guile, MzScheme, Chicken) are supported. SWIG is most commonly used to create high-level interpreted or compiled programming environments, user interfaces, and as a tool for testing and prototyping C/C++ software. SWIG can also export its parse tree in the form of XML and Lisp s-expressions. SWIG may be freely used, distributed, and modified for commercial and non-commercial use.
---

Or in other words, Swig takes a C++ class, and creates a wrapper to make that class usable from any of the (interpreted or compiled) high level languages above. Which one you prefer is another issue. Needless to say, it would be almost impossible to have such a coverage of available "glue-language" support otherwise.

Another project which might be a source of inspiration is this one:

http://www.tntnet.org/ - TNTNet is a "C++ Application Server" which already implements a kind of "C++ Server Pages", which allows to compile an entire web application into a single executable without many dependencies (yes, even images and binaries are compiled into this as resources). TNTNet is GPL Software, so don't mix code.

There are many scenarios where this can be a huge advantage. It offers much better runtime performance, ease of deployment, and many advantages in resource consumption.

TNTNet is very nice, but also very focused on a single kind of application, and it really embraces the "JSP" kind of web application development, while neglecting the "Servlet" kind of development. Although it's possible to work around this, I'd prefer Pocos HttpServer, and write some code generator on top. (Performance is comparable, btw - I measured)

When it comes to Code generation and Template Libraries, I would also like to point you to the work of Terence Parr (Developer of ANTLR - a very good Parser Generator which can also create C++ Code, and the StringTemplate Library. Also, Professor of Computer Science in SF:

http://www.stringtemplate.org/article/list

I would especially recommend the first three articles on that page. They make more sense if you are familiar with ANTLR, StringTemplates, JSP and the shortcomings of JSP and most Template engines.

Coming to a conclusion, I would recommend the following course:

- Using Swig or a comparable tool (I think there might be some others) to create wrappers around application classes is not neccessarily a task of the Poco Framework since you will usually want to wrap high level objects as well, not just low level Framework objects/classes. If you would just wrap Poco's classes and don't expect the application developer to do likewise with his applications model, this will evolve into a scripting language like PHP or Python, and I don't think thats what's intended

What I would make sure, though, is ensure that using Swig (or something like it) runs smoothly with the Poco Framework. (No hiccups parsing important classes like HttpRequest, HttpResponse and so on).

- I would add a template based code generation framework to Poco, which (using a custom parser, maybe built with ANTLR) allows to recursively expand code templates to actual C++ Classes which are to be used in some context. (For example as "C++ Server Pages") - I would not allow flow control statements in the top level template (component definition), but allow arbitrary C++ (or scripting language) code in lower level templates (component implementation). Recursive expansion taken alone is a very powerful, secure and proven method for code generation (the C Preprocessor uses it, it can't be that wrong :).

- I would also add a few classes to integrate a C++ compiler to compile generated classes at runtime. I.e. a generic interface to incorporate specific compilers such as C++ into the runtime model, especially allowing the creation of shared objects.

Well, I posted this in the wishlist area, but it might well be that I am going to implement some part or another of this, and contribute it back to Poco once it works. Unless you are faster, or stop me of course :)

Feel free to contact me directly, anyway ..

best regards,

Kai

[klondenberg]

> But apart from that, scripting capabilities are often a neccessity. I see pre-compiled pages as a complementary, not as a contradicting technology to generated code.

meaning "I see pre-compiled pages as a complementary, not as a contradicting technology to scripting". of course :)

[alex]

Kai,

thank you for the post and suggestions.

> I'm new in this community, so just as an introduction, as you might see from the following, I have already done a bit of research into this topic before I noticed Poco, so you might consider it valuable what I found and planned.

We value even your interest, let alone when you do something like that. Thanks.

> I would especially recommend the first three articles on that page. They make more sense if you are familiar with ANTLR, StringTemplates, JSP and the shortcomings of JSP and most Template engines.

Let me add this to the mix:
http://code.google.com/p/google-ctemplate/

> Well, I posted this in the wishlist area, but it might well be that I am going to implement some part or another of this, and contribute it back to Poco once it works. Unless you are faster, or stop me of course :)

We have no desire or intention to race you nor can we stop you from doing anything - Poco is free. What we can do, though, is talk to you and see if we can find a way of working together in a mutually beneficial direction.

That being said, let me point out that Servlet is not yet a release-grade code. The thing runs and runs well both on Linux and Windows, but it needs some work before it is released. Since I am currently very much tied up with work and school stuff, it'll probably stay like that for a while unless someone steps up and smoothens the rough edges so we can release the thing. I would personally like to see PocoServlet released before anyone seriously embarks on any add-ons or embellishments development on top of it.

In the meantime, we can keep on discussing the server pages/scripting interface. As you have probably noticed, Poco community is very picky (in a positive way) about the code quality and aesthetics. Other widely-accepted open source projects (zlib, expat, sqlite ...) are leveraged in Poco, but they are tightly integrated and thus transparent to the end user. We believe this is critical for the success - dependencies must not be external, except for those found on virtually any system (such as gmake on a Linux box, for example). You as an end user should not be required to download another dependency yourself. This is mostly, but not completely upheld and there are exceptions to this rule where appropriate. An example are DataConnectors - most of them will depend on their respective DB communication layer to be present on the system at both compile and run time.

I have heard about Swig before but have never used it myself, so I can not give it either up or down vote at this time.
As for ANTLR, it looks like a great tool and I've been looking into it for potential use with PocoData for purposes of gaining deeper control over SQL, but it's a far shot at this time. And again, it would require a serious discussion of whether such path is worth taking.
And then, recently there have been some suggestions/proposals for CppParser expansion to a full-blown thing, but it remains to be seen whether that will go anywhere any time soon.

> Feel free to contact me directly, anyway ..

I can't get through to your email. So, why don't you just drop me an email at aleskx AT gmail DOT com.

Regards,

Alex