Using HyperCard as a CGI application
Some tips and tricks
CGI (Common Gateway Interface) is a standard for the communication of Web servers with other programs, allowing the output of dynamic information. These programs, usually labelled with the suffixes .cgi or .acgi, process data received from the client via the server (e.g., they enter data into a database which have been submitted as a form entry) and/or reply in response to information provided by the client (either by constructing and sending a complete page, or by sending a http address (URL redirection) which the client then requests; the client information is not restricted to user input, you can determine the client software and return a page optimized for the respective client).
Many complete CGI applications are available for different purposes, most of them as Freeware. If none of those can perform the tasks you need, it is up to you to write your own CGI application. If you are a "real" programmer, Perl or C/C++ definitely are the way to go. Then there are AppleScript, much easier to program in (especially when you start with Jon Wiederspan's wonderful tutorial) and Frontier, but their applications are far slower than those compiled from C.
StarNine (makers of WebStar) provides a list of tutorials and examples about producing CGI applications in any of those languages.
And finally there is HyperCard. It is definitely the easiest way for a non-programmer to start with, and while it is far slower than a C-based CGI application, it does not compare unfavourably with AppleScript (to say it cautiously) if you optimize the setup. And if you want your CGI application to link a simple database to the Web, you are much better off using HyperCard's own database abilities (including its fairly fast "find" command) than to program your own database tools. Many tasks can be accelerated enormously when you use an external command (XCMD/XFCN) for the time critical step, and there are loads and loads of these available (it would be unfair of me not to note that AppleScript and Frontier also make use of similar externals).
I know of two ways of communication between WebSTAR/MacHTTP and Hypercard: directly via an Apple event, or indirectly via an AppleScript CGI application which is called by WebSTAR and then uses Apple events to communicate with HyperCard. The only reason to use the indirect strategy I can think of is that something must be done which HyperCard can't do, but AppleScript can (and probably an Xternal exists somewhere which allows to do it in HyperCard), otherwise, the longer chain of programs will run slower and will be more prone to crashes and timeouts. In a word: I use and will describe only the direct communication (and I have no experience with server software other than WebSTAR/MacHTTP, so I have no idea whether the following is true for other Mac Web servers).
To write a CGI application in any language, you must know what kind of data you will have to deal with. Jon Wiederspan's tutorial contains so much basic information about how CGI applications work (and is easy and fun to read) that you should check it out even if you intend to script your Web site using Hypertalk only. He describes in detail what kind of data (arguments) are transmitted and in which format, so there is no need for me to repeat it here.
Overview: What Hypercard has to do as a CGI applicationWebSTAR will send its SEARCH or POST data (you had a look at the Wiederspan pages?!) packed up in an Apple event. In HyperCard this triggers the appleEvent system message which you will have to deal with in an "on appleEvent...end appleEvent" handler. The Apple event can be identified by its parameters class, id, and sender which can (and should) be used to make sure that the incoming event really comes from WebSTAR (don't forget to let other Apple events pass). The enclosed data of the Apple event are extracted with the command "request ae data" (returns the path args) respectively "request ae data with keyword" (keyword "meth" returns the method: POST or GET, "post" returns the post args, "kfor" returns the search args, etc.). Your script now has to parse (extract the data and put them into the correct containers) and decode (convert + to space, and convert the coded characters from %##) these data, then you can process them ad libitum (e.g., enter them into your database or use them to search your database). Finally, produce an answer (either a complete HTML document or a redirect to an existing document) and send it back to WebSTAR with "reply yourAnswer". Because the returned data are passed to the client without further processing by the server, you have to provide them with a header (WebSTAR does that for the http documents it reads from disk before it sends them to the client) which describes whether a document (header contains "200 OK") or an address (header contains "302 FOUND") is returned. Omitting the "200 OK" header spares time and usually gives no problems, just like crossing a street when the traffic lights are red...
The handler has to
Put an alias to HyperCard (version at least 2.1) into the same folder as WebSTAR respectively MacHTTP, and name it "HyperCard.acgi"; put the HyperCard stack with the "on appleEvent" handler ("CGI-Stack") into the same folder. Start HyperCard by dragging the CGI-Stack on the HyperCard.acgi alias. A form which calls the CGI-Stack should contain
- grab the Apple event
on appleEvent class, eventID, sender (...end appleEvent)
verify it is a sdoc event from the server
if class & eventID is "WWWsdoc" then (...else_pass appleEvent_end if; the letter after WWW is a capital Omega, Option-z)
- extract the relevant data
request appleEvent data -- get (and evaluate) the direct parameter (path args)
request appleEvent data with keyword "post" -- post args (...put it into postArgs)
request appleEvent data with keyword "kfor" -- search args (...put it into searchArgs)
- the post args will have to be parsed and decoded
- set the itemDelimiter to "&"
repeat with Element = 1 to the number of items of postArgs
if char 1 of item Element of postArgs is "I" then
put URLDecoder(item Element of postArgs) into Input
put empty into char 1 of Input -- remove inputName=; I use 1 letter names
put empty into char 1 of Input
... repeat for all possible elements
- while the search args have only to be decoded
put URLDecoder(searchArgs) into searchFor
- do whatever you want to
(search with them, add them to a database, control other programs, use them to direct your video camera...)
- produce either a complete http document or redirect to another http file
- put Header & Sheet & Footer into answerDoc -- here is some work to do: it is your business to mingle the data you want to display with the appropriate HTTP tags
- send it back to the server
- reply answerDoc
<form method=POST action="HyperCard.acgi$PathArgs">
the form input is put into the post args; a direct link to the CGI-Stack looks like
This basic setup has several weak spots. If you forget the unusual opening procedure or if HyperCard quits for some reason, the call from WebSTAR will fail and produce an error. The CGI-Stack also will not be opened automatically on startup. The CGI-Stack must be the frontmost HyperCard stack, otherwise the Apple event is sent to the wrong stack, resulting in an error. And while any other script is running in HyperCard, or while you are editing a script, the execution of the CGI-Stack is blocked.
I found some tricks and modifications to avoid the problems above, and to speed up HyperCard as a CGI application. But still I sometimes find errors like
Error -1700: GetAttrPtr retID (Reply Data could not be coerced to requested type (char))
Error handling HighLevelEvent -1700
Error -1712: Sending CGI sdoc Event (AppleEvent timed out)
Error executing CGI application
in my WebSTAR log window without having an idea of what went wrong. I live with that, and you will probably have to, too, if you decide to use HyperCard as your CGI application (I am sure that programs written in other languages have some bugs, too).
Enough of gloomy warnings, now for my improvements:
I wanted the Hypercard CGI application
Only 5. is something I could do nothing about. However, WebSTAR (or the Apple event manager) seems to handle the problem as well as possible. I suppose/hope that it queues the requests and sends each one off only when the previous has been finished.
- to be fast, or at least not to take forever
- to work without manual help: if not running when called from the server, it should start up correctly all by itself
- to serve several different databases located on different stacks
- to allow me to use HyperCard without blocking the HyperCard.acgi
- to handle several requests at the same time.
- to 1. Speed is definitely a problem in HyperCard, but several things can improve it:
- Do not put an AppleScript between the server and HyperCard. HyperCard can handle the Apple events sent from the server just fine (I use the script example code from Chuck Shottons CGI Demo Stack)!
- Use "set lockScreen to true" at the beginning of your appleEvent handler. Updating the graphics often needs most of HyperCards processing time, and is sheer waste for a background activity. Unless your CGI scripts rely on sending messages, you can further speed HyperCard with "set lockMessages to true". An additional "set lockRecent to true" is not necessary, because ("lockScreen" + "lockMessages") implies "lock recent" (and at least in my setup, "lockScreen" alone prevents additions to the card list).
- For time consuming steps try to find (or write) an Xternal. The most obvious steps are the decoding of the &## to chars and the conversion of the Netscape "+" to space. I found a pair of Xternals (Motoyuki Tanaka's MtDecode from his stack ClipDecoder1.1, and Tracy L. Hinshaw's QuickReplace from Parnassien's Little Page Of Externals (http://www.netins.net/showcase/fornlngz/card/parnassien_externals.html. Since her site seems to have vanished, I have put her stack into my ftp archive)) which handle these tasks fast. You will need ResEdit or some other resource handling tool to move the XFCNs to your stack (or start from my example stack).
It would be nice to have an Xternal for handling the parsing of the post args, as the OSAX "Tokenize" does for AppleScript. If you know of such an Xternal or of a way to call an OSAX from Hypertalk - give me a hint.
The "find" command of HyperCard should be fast enough for most cases, so there probably will be no need to use a replacement. For heavier database tasks there are more powerful Xternals, e.g. the commercial CardTable (contact Karl Petersen for details) which allows to quickly search files on the hard disk in addition to the stack.
- HyperCard is much faster (often more than tenfold!) when it is the application in front. I had always smiled about the flaming of UNIX people against the Mac because of its lack of preemptive multitasking, but now I just hope that System X will work as advertised - and especially, will still work with HyperCard.
Microsoft Word running in the foreground can slow HyperCard in the back so much that the request times out before it is fulfilled. The same happens when an alert pops up (in my case a "You have new mail" blocked CGI execution for a night, I have permanently switched off the alert and rely on a blinking menu bar alone for notifying me - one more excuse for my slow replying to e-mail :-). So whenever you are not using your Mac WWW server for something else, make HyperCard the frontmost application by calling it from the application menu in the upper right corner of your menu bar (advice from both Cindy M. Carney and jda):
if the suspended is true then
doMenu "HyperCard.acgi" -- Use the name of the program exactly
end if -- as it appears in the menu
Called right at the beginning of the appleEvent handler, this pops HyperCard to the front, resulting in a fast processing of the request. It also works if you have two copies of HyperCard running in parallel (see below).
Of course, if the Web server is your normal desktop machine (as in my case) and gets lots of CGI requests (not true for me, not yet, at least), this application switching could become very annoying.
If you want to restore the frontmost application after the CGI action, you must check for it at the very beginning of your appleEvent handler while HyperCard is still in the background. I extract this information from the first item of the application menu:
get line 1 of menu "Application"
put empty into the first word of it
put empty into the first char of it
put it into OldFrontmost
and at the end of the handler
(In my German version, the first line of the Application menu is something like "HyperCard.acgi ausblenden", so I delete the last word and the last char of it; I suppose that in the English version it is "Hide HyperCard.acgi": correct me if I´m wrong.)
Because processing must take place in the background it needs some time: around 120 ticks/2 seconds on my Centris 650, and significantly more if the frontmost application is very busy or egoistic. So, if speed is your primary interest, leave HyperCard in front or switch to WebSTAR at the end.
- to 2. The server calls HyperCard, not a specific stack (starting the program when it is not already open). And starting HyperCard results in opening the stack Home. So the solution is to put the "appleEvent" handler into the script of the Home stack.
This also helps in case something is messed up and the wrong stack is in front. The frontmost stack always gets the appleEvent message, stacks in back can only take over (if the front stack can't handle the event) if they are "in use". The Home stack is "in use" by default, so it will receive the appleEvent if the stack in front can't handle it.
Another situation which can block CGI execution by requesting human interaction is the occurrence of an error which pops up the Debug/Script/Cancel dialog. Apple Script allows to trap most of those errors with "try ...on error". Wouldn´t it be nice to have such a security net in the next version of HyperCard, too?! Until then, it is up to you to test your CGI-Stack with all possible inputs. Do not rely on the options you offer in your forms and links. By accident or ill will, anything could be sent as path args, post args, or search args (hint for the would-be hacker: save the source of a form, modify the "methods=" part and try it out ;-)! Some important things to test are how your CGI program behaves when the input field is empty, when it contains a "return" (line break), when it contains one or more "quotes", and when it contains a long string of chars (255 characters is a critical size, the do and find commands choke on strings above that length)!
- to 3. The Apple event sent by WebSTAR is addressed to the application specified in the anchor or form, which will be addressed by its name. There is no way to directly address different stacks (well, I could think of a bizarre way to do it: open the different stacks with different copys of Hypercard, which might be named Hyper1.acgi, Hyper2.acgi..., and you can address the different stacks by calling the respective copy of HyperCard).
My solution to serve different stacks: Use the Home stack as the distributor. Its "appleEvent" handler extracts the relevant information from the Apple event (in my handler the path args (appleEvent data) and either post args (appleEvent data with keyword "post") or search args (appleEvent data with keyword "kfor"), decides from the path args which stack is requested, moves it to the front and calls a function (which can also vary with different path args, so you can perform different actions with the same database) with the post args respectively search args as the parameters. The function fullfills the request, builds a complete HTML document and returns it to the appleEvent handler, which sends it to WebSTAR as the reply.
If the requested stack has not been running, it will be opened, but closed again after it has handled the request. Each opening takes time, so better make sure that all stacks which might be called are running (by calling "go "stack" in new window" (with the names of the respective stacks instead of "stack") for each stack in an "openStack" handler in a background script of the Home stack (not in the stack script, otherwise it is called whenever a stack is opened which does not contain an openStack handler)).
- to 4. Using HyperCard without blocking the HyperCard.acgi? Impossible, isn´t it? What a pity, especially because using HyperCard for WWW serving indicates that you are using HyperCard a lot. The solution is trivial: don't put an alias to HyperCard called HyperCard.acgi into the WWW folder, instead, duplicate HyperCard, put the copy into the WWW folder and call it HyperCard.acgi.
And, put a copy of the Home stack into the same folder (only that copy of Home needs the "appleEvent" handler, and I stripped it of most of its "openStack" handlers to speed the start of the program). To use the other copy of HyperCard in parallel, drag the stack(s) you want to open on its icon. I only hope that this solution does not interfere with the licencing conditions for HyperCard?! If you want to edit a stack used for serving, edit a copy and replace the original when you are finished.
My use of HyperCard as a CGI application
What I set up with HyperCard is the following (have a look to see how well or how poorly HyperCard behaves as a CGI application. Caution: the server is my desktop Mac (a Centris 650), so a slow response from 8am to 8pm MEZ (2am to 2pm Eastern Time) is probably due to my foreground activity and System 7's lack of preemptive multitasking, slow response at other times can only be blamed on HyperCard and my programming):
- A search form for the database of a plant family (Ancistrocladaceae) producing interesting compounds (naphthyl-isoquinoline alkaloids):
search for, e.g., korupensis or hamatus.
- The numbers listed as "Contained compounds" in the answer page link to a database of the naphthyl-isoquinoline alkaloids (method GET).
- The same database can be searched directly with a search form
(search for, e.g., Droserone or Plumbagin).
- The AAA protein superfamily can be searched with a search form
(search for, e.g., CDC48 or SEC18 as the name or choose a family from the select menu).
- The answer page contains links to the same AAA stack to extract the complete data for the respective protein (method GET).
- The AAA stack is also addressed from a list of members of the protein superfamily and from a clickable map, MapServe.acgi returns a link to HyperCard.acgi (method GET).
The same data are available from a list and a clickable map whose links return files from the hard disk instead of letting them be created on the fly by HyperCard.
These last two pairs of URLs provide a fair comparision of the access times (same computer on the same line, identical data) and show how much HyperCard adds to the reply time. Our computer center through which all our transmissions are channelled often has a very slow connection to the rest of the world which then affects both "hypercarded" and direkt HTML transmissions. (Because these last two HyperCard links are purely experimental, they are not linked to from elsewhere in my server, not even from the files they call, so use the Back function of the browser for multiple tests).
- A bulletin board for student groups at our University. It allows the posting, reading, modification, and deletion of news. Posting is restricted to the local domain (Universität Tübingen), modification or deletion is only possible for the originator of the respective news. Expired news (the expiration date is set by the originator up to a maximum period) are deleted automatically. The programm is based on my example stack of a HyperCard based CGI database.
Other sites using HyperCard as CGI application/HyperCard CGI ExamplesIf you use HyperCard to serve the Web, tell me and I will provide a link here:
- Ambroise.B uses HyperCard on his Infolipo site to serve pages about "Informatique & Littérature potentielle" (BTW, his inquiry on comp.sys.mac.hypercard incited me to produce this page).
- Lars-Olof Albertson, author of the HTML-HyperEditor, has just recently made a form-processor to import form fields from a tab-limited text file to FileMaker Pro or to HyperCard, the reply to the client echoes the textfields just entered, and a one year calender available for everyone (...who understands some Swedish). He is working on a stack to be used as a discussion forum.
- PrintNET Inc. uses HyperCard CGI stacks for a number of services, including their Industry Database which you can search for links to printers, designers and publishers, or add a link to your company. Version 1.0 of the CGI programm has been made freely available by author Joe Warmbrodt (see below), and further versions hopefully will be, too.
- Traci's Place is a WWW chat room run by Ronald Vullo based on his Hypercard stack Traci which he makes freely available (see below).
- Rob Toreki lets you test your knowledge of the periodic table of elements and has you graded by a Hypercard CGI stack.
- If you think that HyperCard is only suitable to serve small databases, you should have a look at Dan Wengers HyperCard CGI project. His web-searchable genealogy database consists of more than 56,000 cards!
- Dario Gibellini used SuperCard, HyperCards younger brother, to create his Shareware Chat Forum CGI MacChat Light which is used at, e.g., the University of Lüneburg. He has written more Hyper- and SuperCard based CGI programs and uses Heizer/Royal Software's Double-XX to convert them to compact programs.
- LiangTyan Fui has produced a set of Xternals expecially useful for writing Hypercard CGI programs.
- Edwin Haskell uses Hypercard for online calculations of mass energy absorption coefficients for complex molecules, and for a scientific reference with 16,000+ references that can be searched.
The best way to learn programming is to look at good code. If you are serious about setting up HyperCard for CGI handling, you should definitely download and investigate the following stacks:
- Chuck Shotton's Demo CGI Stack
- Dennis Birch'sVisitingTime.acgi (which also is a great example for the use of WebSTAR Actions)
- A page with several examples, provided by the Academic Achievement Division
- The hypercgi stack from Joe Warmbrodt (available from info-mac mirrors as /comm/inet/web/hyper-cgi-hc.hqx) includes an elegant way to replace symbolic tags by their actual values (e.g., <!time> is replaced by the actual time) on the fly during the HTML request and other code definitely worth a closer look.
- Ronald Vullo's Traci contains an ingenious way to extract the field variables from the post_args, XFCNs to decode URL-encoded characters and to replace acronyms by inline pictures, and much more.
put numtochar(13) & numtochar(10) into crlf
put "HTTP/1.0 200 OK" & crlf & "Server: WebSTAR/1.0 ID/ACGI" ¬
& crlf & "MIME-Version: 1.0" & crlf ¬
& "Content-type: text/html" & crlf & crlf & "<HTML><HEAD><TITLE>" into OKHeader
go "Stack1.acgi" in new window
go "Stack2.acgi" in new window
-- call all stacks addressed by CGI requests
function URLDecoder myData
-- Calls the Xternals MtReplace and MtDecode
if myData = empty then return empty
put MtReplace(myData,"+"," ") into myData
put MtDecode(myData) into myData
put MtReplace(myData,(return&lineFeed),return) into myData
put MtReplace(myData,lineFeed,return) into myData
Please, mail me suggestions for improvement, corrections, info about interesting links.
[To our home page][Our lab and projects][To the software archive]
Last edited: October 23, 2001 by KaiFr