Friday, August 31, 2007

Use Case for Downloading: An Intranet Web Site

A large amount of Web development is focused inward and is used to power company portals, content management systems, and myriad other applications. These applications are perfect places to deploy AJAX because they usually provide a high-bandwidth, homogeneous environment. The homogeneous environment aids in testing because you have fewer browsers to test against. In addition, the high bandwidth allows you to pull in any tool you need without much fear of increasing download times to an unacceptable level. These characteristics open up a huge number of possibilities when choosing an AJAX library; you may even choose to combine several to get the mix of features and the APIs that best suit your needs.

Internal sites generally have high levels of interactivity and may be the main application used by large numbers of employees. This is especially true within many content-management systems. They have many areas that can be enhanced through the use of AJAX, especially in the editing process. Commonly, content-management systems add in AJAX-based autosaves to keep authors from losing content. AJAX-based editing that allows for quick processing is also useful. Any powerful application will have many different places where AJAX makes sense, and these areas will have a variety of different communication patterns. The applications may also be enhanced by a variety of additional features. For instance, a content-management system may find drag-and-drop ordering of articles to be especially useful. Any application may also benefit from graphical fade effects that notify users that an action has taken place.

As you can see, an intranet Web site needs an AJAX library or multiple libraries that provide many types of features. The library needs to provide a communications layer that tightly couples with the backend programming language and framework while also covering different development patterns. These patterns range from buffering search requests on a find-as-you-type system to providing timed updates on an autosave system. Then, the library needs to provide graphical effects that can enhance ease of use; these can include features ranging from a visual effects library for fading in new HTML elements to a drag-and-drop ordering system. Writing a library that provides all these features can be a huge undertaking; this makes looking to prebuilt libraries a great solution.

Read more...!

Use Case for Building: The Firefox Counter

After the Firefox 1.0 release, the Mozilla project added a number of grass-root marketing approaches to their marketing effort. One of these was asking people to add a counter to their Web sites showing the number of Firefox downloads. This data was provided by a Really Simple Syndication (RSS) feed; some implementations read this data at the server and then rendered it during the normal page generation process. Building on this basic approach, Matthew Levine built a small AJAX odometer-style counter that updates continuously. An example of this is shown in Figure 4-4. In this figure, an AJAX request is made on a regular interval, with rate information being used between updates to continually update the displayed count.


This is a very basic use case for AJAX. To implement the counter, you need to do a GET request to a single page and then grab three data points from an XML file. Because you're only going to be making a single type of request, you don't need the full-blown AJAX support that most libraries provide. It's also a small feature you are adding, so lots of error handling or other status feedback isn't really useful. The concept behind the counter is that it's an informative marketing technique, and while it's more active than a mere image, it's not the major draw of any page it's on. To accomplish this design goal, all you need is an implementation that updates in a smooth fashion or that dies silently when there are problems.

Simple functionality, combined with the goal of widespread installation on many different Web sites, makes building custom AJAX implementation a good choice. In the case of the actual counter, there is a small PHP script that produces the feed data; a small bit of JavaScript code; and a small HTML page that can be used as an IFrame to tie the pieces together. This makes for easy installation and a small amount of additional weight for the target page. The ease of installation and the small size are important features for any code you're hoping to widely deploy in a marketing effort.

In this case, building custom AJAX code was beneficial because ease of deployment and small size were more useful than the quick development time that a pre-built library might have offered. The simplicity of the AJAX code also removed much of the need for a library because in this basic case, we needed to focus on only one type of request. If we were combining the counter onto a site already using AJAX, we might want to swap out the communication component to increase consistency, but because a counter by itself needs no other features, it makes little sense to bring along other features that will never be used.

Read more...!

Thursday, August 30, 2007

How Open Source Fits into the Mix

As you look for an AJAX library, you will notice that many of the most mature options are open source libraries. In fact, most of the first libraries were open source, and an ecosystem has grown up around them; this has made it much harder for high-priced commercial libraries to gain a following because they have to offer more than their free open source counterparts to get people's attention. Open source AJAX has also been driven by its match with open source scripting languages. These languages include Python, Ruby, and PHP. These languages often pick up new technologies faster than vendor-driven languages, such as ASP.Net or Java, because there is more competition at the tool level. This has allowed for quick integration of AJAX, although the implementation isn't always mature.

Evaluating an Open Source Library

When picking an open source AJAX library, keep in mind three main items: license compatibility, feature set, and maturity/community size.

License compatibility is simply a matter of picking a license that meets your needs; if your software will be released as open source, then the licenses need to work together. The simplest solution in this case is to pick a library with the same license as your currently chosen code or one that is very liberal, such as Berkeley Software Distribution (BSD) or Massachusetts Institute of Technology (MIT) software licenses. If you're using the library for internal development, licensing isn't generally an issue because most licenses apply to distribution, and you're the only user. If you're selling the software, you'll need to be more careful because you'll need to pick a license that has redistribution rules you can deal with. Open source licensing can be complex, but not necessarily more so than managing licensing from commercial vendors.

As with any library, picking one with an appropriate feature set is of vital importance. You may be quick to pick one that offers the biggest checklist, but it's generally a good idea to look at a couple options and pick one that fits your coding style and that can be easily combined with other libraries. As with any development, there is always a tradeoff between large monolithic libraries and smaller components. In addition, be sure to pay attention to how well the library integrates into any Web-development framework you might be using; an AJAX library designed with your framework in mind can, in many cases, provide a simpler development experience than a more general library.

The last item to look at when picking an open source library is the maturity of the project and the size of the community around it. The community offers support, testing, and documentation. The larger the community, the easier time you'll have getting started with the project. A large community also reduces risk because there are more people to share development in case the project's original developers abandon it. Because the barrier to entry in starting a new project is low, you'll want to pick a project in which the developers have proven they know how to support their users over more than one release. Libraries associated with larger projects, such as the PHP PEAR project (HTML_AJAX), or a framework like Ruby on Rails (Prototype), are also good picks because they have a large infrastructure and knowledge base from past larger projects.

Open Source Libraries in Relation to Commercial Libraries

Open source is popular in the AJAX world because it offers low cost, ease of customization, and widespread support in every possible language. Mainstream mature projects offer a quick route to AJAX deployment and a large community that is a great source for support. Smaller, more experimental projects can also offer great value if they meet your unique needs, but you need to be careful because many small projects will never generate a large enough community to gain support beyond their original developers.

Commercial languages developers, such as Microsoft, have also started developing AJAX libraries. These libraries have many of the same distribution advantages as open source libraries because they are not tied to per-server licenses. However, they don't offer the same customization possibilities that an open source library does. They also lack the ease of developer interaction that most open source projects have. This lack of ease occurs because there are fewer lines of communication with the developers of the actual library. The biggest advantage that libraries from major vendors have is the resources behind them. This leads to widespread testing and thorough documentation, even in early releases.

How I Decide Which Open Source AJAX Library to Use

The company I work for, Uversa Inc., is based around General Public License (GPL) software, so when I pick any library, it first needs to be compatible with the GPL. Because the GPL is so widespread, many licenses are compatible with it. (See www.fsf.org/licensing/licenses/index_html#GPLCompatibleLicenses for more information.) However, because licensing is a hard rule, you should always start your search by limiting it to the ones that meet your needs. After getting my license guidelines, I look at major features that are required. In my case, this includes good compatibility with PHP, including the ability to map data types between PHP and JavaScript. I also want to be able to easily combine the library with other JavaScript libraries, so well-name-spaced functions and variables are a plus. Finally, I want a focused design, so I'm looking just for an AJAX library; I don't need a large JavaScript framework that takes weeks to learn. Multiple developers will be using it, so the less they have to learn, the better.

During most of 2005, these requirementsand a bit of searchingwould have left me with a small list of libraries from which to choose. I could investigate them and find one that fit the rest of my Web development framework without too much hassle. Today, though, these requirements leave me with a large list, so I need to enter some other items to narrow the list of items I'll investigate thoroughly. I can further limit my list by picking projects that are actively being developed, so I'll look in depth only at those with releases in the past few months and that seem to be developed by more than one person. You don't want to remove every single-developer project (after all, that's where many of the most innovative ideas come from); you just want to make sure that enough releases have been made that the library is not a one-time code drop of unfocused ideas. These criteria will help weed out the unsuitable projects and will keep me from wasting time on a project that will never gain the community needed to sustain it over the long run.

Once I have a short list of libraries, I'll do a quick review. Everyone has different goals, but I like libraries with at least basic documentation and an object-oriented (OO) design. (OO design is especially important to me on the PHP side because it will need to mesh with my existing code.) A good way to test any library is to do a basic install and to build a basic "hello world" application with it. If you can't easily complete a basic task, then the library probably isn't a good fit. AJAX isn't such a complicated technology that the basics can't be made easy while still making the difficult items possible.

Hopefully, after some basic use, one of the libraries will stand out from the pack and end my search. If a few libraries seem really good, I'll dig further into their documentation and user forums and make a final decision based on how easy learning all the details will be. If none of the libraries looks like it will work, then I'm left back at the starting gate. I can expand my search and look for less popular and hence harder-to-find libraries, or I can look into developing my own solution.

In my earlier searches, I had very few options when I was selecting a library; my first foray into AJAX was before the term had been defined. I picked the JPSpan library for its good PHP integration and object-oriented design. Although JPSpan was a decent solution, it didn't end up meeting all my needs. Over time, I decided to develop my own library, HTML_AJAX, for PHP's PEAR project. The reasons for building my own library relate more with wanting to help the PEAR community than in meeting my needs, but once you have your own library, it's an easy front-runner for future use.

As you make a decision on what library to use, you can apply much of this process. First, decide on your licensing needs; your needs can be as simple as a specific open source license or as complicated as a commercial solution. After that, look at your feature requirements, especially server-side language support, and build a list of possible solutions. If the list is large, looking only at more active projects is a great way to pare down the list. Then, take some time to investigate the libraries. I find it's always worth my time to actually write a small amount of test code. After that, it's just a matter of picking a library that seems like a good fit. Don't forget to take into account everything into which you'll be integrating this library; some solutions that might be easy in a standalone situation become a bear when integrated into your server-side Web development framework.

Read more...!

Wednesday, August 29, 2007

Reasons to Build Your Own Library

There are two reasons to build your own library: control and lack of a good alternative.

Control is the most common reason to build your own library; you want things to work in a specific way. Usually this is a matter of making AJAX fit into the ideas of a current Web development framework. You might also build a library to get a specific set of features, although this need generally presents itself only when minimizing library size is also a goal. This lack of feature want occurs because many of the large open source libraries already have such a large feature set. The need for control is often centered on intellectual property. There are cases when owning the copyrights to all your development is more important than cutting down the amount of work you need to do. In these cases, you're forced to start from scratch, although generally it is a good idea to look at libraries with liberal licenses because they offer many of the same intellectual property benefits as writing the code yourself.

Sometimes, you may also find that no AJAX library meets your needs. This is more likely to happen if you're developing a project on a nomainstream language. It can also happen if your project needs low overhead in terms of code size and has limited feature needs. Few AJAX libraries go the minimalist route because it's hard to meet a large number of people's needs in this fashion. Most single-purpose AJAX code supports only a single type of request and has little to no configuration. This is great if the code is written for a single application, but it generally isn't useful for a wide range of development tasks. Building your own AJAX library makes sense for many projects, but don't underestimate the amount of work that building a library takes. While the initial development may be easy, tracking down browser bugs in older browsers, or browsers with a small market share, is a time-consuming process, and it's this widespread compatibility provided by these workarounds that maximizes the cases where you can use AJAX.

Read more...!

Monday, August 27, 2007

Advantages to Using a Library

As with any other technology, you can grab off-the-shelf components to handle implementing AJAX instead of writing all the code yourself. This can help reduce the changes to your overall development cycle, or it can cause more changes than starting everything from scratch. This effect depends not only on the quality of the library, but also on its fit into your current processes and development style. If you can find a library that fits your needs, it will help reduce the changes wrought by adding AJAX to your toolbox. The largest advantage comes from the library's ability to hide the hard parts of JavaScript development (cross-browser support, communications between different languages, and visual effects), but you can also make gains just from following the best practices of someone else instead of having to spend time figuring out all the new rules on your own.

When developers first take a look at AJAX, they may think it's a simple technology with enough support from major browsers that they can take off coding from scratch. This approach ignores the fact that even the major browsers have implementation differences around which you're going to need to work. If your approach is too simplistic, you can end up in a position where it's impossible to work around various browser bugs without recoding large amounts of your application or Web site. The better that the basic browser primitives are hidden, the easier it is to plug in new compatibility techniques. These techniques range from using IFrames on older browsers to implementing new JavaScript API elements that haven't yet been added to browsers.

An AJAX library needs to cover basic communications aspects, allowing you to make asynchronous requests to your server from all the browsers you need to support. Depending on your choices, you will also need some other features; if you need eXtensible Stylesheet Language Transformation (XSLT) support, you'll need a library that hides the differences between the Firefox and IE implementations. You may need code that encodes data in a specific type to be sent to your server; this can be any format, from JavaScript Object Notation (JSON) to Extensible Markup Language (XML). It is also useful to support the tasks often used in conjunction with basic AJAX development, including drag-and-drop, visual effects, dynamic positioning, and DOM manipulation.

A good library helps hide the problematic areas of JavaScript, adding in compatibility between different browsers while offering a clean, easy-to-use API. It also exposes its feature set without making you learn every inch of its API. Drag-and-drop is a great feature, but if it gets in the way of something more basic, then it's making tradeoffs that aren't useful to a developer. No matter the source of your AJAX library, make sure it meets your needs in a clean well-encapsulated manner. Don't be afraid to mix and match libraries, taking the best features from each one, but remember that each library has a cost in download size, and not every environment will be a high-bandwidth local network.

Read more...!

Problems Created by the New Development Paradigm

Most of the problems with AJAX are caused by its added complexity. Because the communication with the server happens in an opaque manner, it is easy to lose error messages or learn that there is an error without also getting all the additional details you relied on before using AJAX. These aspects force you to rely on server-side logging, which is something most casual developers seldom do. The complexity is also increased by the more frequent use of JavaScript. No matter how comfortable you are with it, you still have two main development languages (JavaScript and whatever you run on your server) and cross-browser compatibility to worry about.

There are a number of ways to manage this added complexity. One of the simplest solutions is to create test constructs that allow you to test your server components without connecting them to the front end. The additional complexity created by adding a new language is harder to manage, but it can be done, either by using developers with lots of JavaScript experience or by limiting the scope of your JavaScript development to only those features that have the biggest payoff.

Developing with AJAX can also create new usability and design problems. As you add more interactive features to a Web site, it moves further away from the model your users have used. One way to avoid this is to make the site look and feel more like an application; this gives the users a clue that the site will be working like an application and not like the Web site to which they are accustomed. However, in many cases, there is no easy way to mimic native applications, especially in areas where no similar native application exists. In these cases, you'll have to use other cues to create appropriate usability expectations from your users. Following the usability guidelines provided in Chapter 6, "Usability Guidelines," can help solve many of these basic usability problems.

AJAX can also cause problems because it is new and because it's a prime candidate for overuse. AJAX is powerful and can create some great solutions, but that doesn't mean it can solve every problem. For instance, you may have a general usability problem that can be solved only by updating the user interface. Throwing AJAX at the problem isn't going to solve anything. In other words, keep in mind that AJAX isn't a magic bullet; to use it effectively, you must keep your goals and overall usability in mind when adding it to an application's design.

Read more...!

JavaScript as a Primary Development Language

JavaScript is a powerful scripting language, but deserved or undeserved, it has gained a bad reputation. If you take some time to look at JavaScript with a fresh eye, you will notice that most of its problems no longer exist. The core language is now standardized with the European Computer Manufacturer's Association (ECMA) standards group and is supported on all modern browsers. Of course, these browsers also support older proprietary syntaxes, and you should avoid these as much as possible. Keeping to the standardized interfaces, JavaScript is portable with a minimal amount of testing and browser-specific code. Because of this standardization, writing complex JavaScript, which was close to impossible in the Netscape 4 days, is now an easy task, although each browser will still need its own testing.

High-quality libraries help reduce the amount of JavaScript you need to write. Many libraries, both open source and commercial, are immature, but the more popular ones are already usable tools, even though it can be harder to find documentation and examples for them than for server-side libraries. JavaScript libraries are especially useful for complicated user-interface elements, such as drag and drop. However, with less-complex elements, such as AJAX communications or visual effects (such as fading an element out), they are less useful because you still have to write all the glue, and that's a large part of the overall code. As AJAX becomes more popular and libraries mature, more and more solutions will be created that will generate all the JavaScript for you, allowing you to handle all the details from your primary development language.

JavaScript's greatest advantage is that it runs directly on the client, so it can react immediately to the user's actions. This interaction allows a JavaScript-driven Web application to offer a highly interactive user experience. The experience is interactive because tasks such as reordering a record no longer take an entire page reload. This direct interaction has driven the development of the language, focusing it on interacting with the HTML DOM. JavaScript's ability to add functions to elements of the page at runtime provides a different programming experience than most other languages. However, its position in the browser gives it the unique opportunity to provide compelling user experience opportunities, especially when teamed with the server communication opportunities that AJAX provides.

Just as with any new language, JavaScript will seem more familiar once you've used it on a couple of different projects. In most cases, the biggest problem isn't dealing with the language, or even the differences in its implementation between browsers, but dealing with the new development paradigm that AJAX brings. Splitting your application into two partsone written in JavaScript and the other written in your normal server languageisn't without costs or problems.

Read more...!

Thursday, August 23, 2007

Integrating AJAX into a Framework

Whether you're planning to add only a few simple AJAX features or use AJAX throughout your site, integrating it into your current Web site design is a must. The more formal the framework, the harder the process isespecially if your framework provides a front controller that is heavily optimized for generating HTML. Frameworks without a front controller have an easier time incorporating AJAX because they can add a new entry point just for AJAX; many AJAX Remote Procedure Call (RPC) implementations provide code to help do this.

The way you integrate with a front controller depends heavily on the style of AJAX you're performing. If you're taking a document-centric approach, integration is generally easy; you just need the ability to create pages in the needed output format. (The controller's normal name spacing will work just fine.) This may take some new development, depending on your current design, because you'll need to generate small chunks of HTML (or other data formats, such as XML) instead of full pages. You will also need to make some naming decisions, such as whether you are going to put your AJAX pages next to normal pages or into their own distinct namespace. A distinct namespace makes it easy to locate your AJAX code, but it divides the code by usage instead of by function, so you can't see the AJAX code's relation to its non-AJAX version. Adding in AJAX pages next to your normal code lets you see the relation, but it makes it much harder to identify all of an application's AJAX-entry points. Either option can work well; the most important point is to use a consistent approach.

RPC AJAX implementations have the hardest time integrating with a front controller. This difficulty occurs because most RPC implementations are focused on exporting classes to JavaScript and have their own mini-controller implementation, which maps incoming calls to these classes. Many also generate JavaScript, which should be added to the page using a JavaScript include, which again needs its own basic controller logic. There are three main tasks you want to accomplish when performing this integration: managing what functions are exported to JavaScript, managing the permissions on those functions, and creating a clean entry point that fits the style of the current application.

The last task is generally the easiest to achieve. With most RPC libraries, you'll be passing information specifying which class and which function to call to the server. This information is similar to the section and page information that most controllers already manage; it allows for a pass-through or mapping system to be created easily. The problem comes with the first two tasks: If you enforce permissions at the controller level of your application, you may find yourself with no other choice but to create tons of stub functions to create the namespace needed for permission enforcement. The final task is deciding which functions to export. The simplest solution is to create classes that are used specifically for AJAX integration, but you may find that mapping functions on your current controllers is a better solution for you. If you need to perform complex permission or partial controller mapping, make sure to choose your library with that in mind. Some enforce strict name mapping between the server and JavaScript side, and most approaches like this will need a virtual mapping of the methods instead.

If you start using large amounts of AJAX in your application, you'll also want to look at ways to standardize your management of JavaScript code. Your framework will need a way to map the JavaScript that is needed to power each HTML page. In a small application, it can all be stored in a single file, but in most frameworks, you'll have various chunks of reusable JavaScript to manage. One way to manage this is to output all the needed JavaScript for a page through a dynamic page on the server, sending headers to allow the client to cache the JavaScript as if it were static. Another option is to build packages of prebuilt JavaScript files and then include the set you need for the page in question. Large amounts of JavaScript development will affect your framework in other ways as well, because JavaScript can become just as important as your server-side language.

Read more...!

Wednesday, August 22, 2007

Changes to the Development Cycle

There are a variety of ways to use AJAX when integrating it into Web applications. On one end of the spectrum, you can use it to enhance a current site, making small noninvasive changes to an already completed application. On the other end of the spectrum, you have applications that are heavily driven by JavaScript and won't work at all if the user's browser doesn't support AJAX.

Moving to a 100% AJAX application usually isn't an option for a mass-market Web site because browser compatibility issues will cut out too many possible users, but it can be a successful choice for internal projects in which limiting support to one or two browsers is possible. Although browser compatibility can limit the possible choices, it's not the most important factor when deciding how AJAX will be implemented. Developer familiarity with JavaScript and the willingness of developers to make changes to the way they normally develop will usually play a larger role in how AJAX is used.

The more you rely on AJAX, the larger the changes will be. In an enhancement-only scenario, the largest impact will be the additional testing that is needed, whereas in a 100% AJAX application, all aspects of developmentincluding design, implementation, and testingwill be affected. The amount of testing needed in any scenario is also affected by the number of browsers that need to be supported. In single-browser scenarios, testing can be focused solely on the new AJAX widgets and user interactions, whereas in a multiple-browser scenario, differences between browsers need to be tested as well; if older browsers are supported, then fallback scenarios also need to be tested.

Enhancement-Driven Changes
AJAX is typically used to enhance a current Web site after the site has been developed to a fully functional state. Specific areas where AJAX could improve user experience are chosen and then optional code is used to add AJAX to those areas. Examples include adding an AJAX-driven pull-down element to a search box that shows matching options as the user types or providing a way to validate that a username isn't already in use without submitting an entire form. Many AJAX libraries encapsulate all the functionality needed for features like these, making the only implementation and design challenge the integration of a new library. While this can be difficult depending on the framework in use, it's not really any different from supporting any other third-party library. If an AJAX library is not used, any implementation will make some major changes to your normal application development cycle; be sure to set time aside to create the needed infrastructure pieces.

Large amounts of JavaScript development can be especially disruptive for groups using unit testing or other automated testing tools. The disruption is caused by the lack of JavaScript support in normal testing tools. Unit testing of JavaScript is possible, but it may be hard to integrate into existing testing frameworks because it needs to run in a browser. In addition, it may need to run in different browsers to cover differences between them. Although getting JavaScript unit tests integrated into an existing framework can be a difficult task, it does have a large payoff because it helps hide many of the debugging differences that AJAX adds.

You'll find that even bringing in a small amount of AJAX leads to huge differences in debugging. These changes are caused by two main items:

  • A larger amount of logic is encapsulated in JavaScript, which creates a second area to test code for problems. This testing process can be further complicated by the need to learn new tools because few Integrated Development Environments (IDEs) support JavaScript. This leads developers to rely more heavily on debugging tools that are built into browsers.

  • The communication process is hidden. When you are loading a normal page, logic errors and debugging messages can be immediately shown, but with AJAX communication, these errors need to be caught in the JavaScript code and handled before they are visible. This makes logging problems at the server level more important and removes the direct feedback that most Web development languages provide.

When adding any amount of AJAX to a Web application, you can expect longer development and testing cycles. If AJAX libraries are used, the amount of additional time needed is about the same as adding any other feature. If all the development is new, then testing time will be increased, as more testing of basic functionality will be needed. Because more features are being added, implementation time is increased, but no other large changes are introduced. (These bolt-on features require little JavaScript to be coded once the basic infrastructure is in place.)

AJAX in Action: Removing a Popup User Search

A common task in many Web sites is selecting a user on which to perform an action. This task is especially common when dealing with permissions systems, where you can spend a large amount of time selecting users to add to groups or to update the permissions on. If the site contains only a small number of users, this can be accomplished by an HTML select box, but after a hundred or so users, this becomes less useful. It becomes less useful because of the time it takes to scroll through the list and find the item in question and because of the large amounts of data that need to be transferred for each box. One solution is to provide a link that opens a new page where the search can be done; the new page would then return the selected user. Although this scales to large numbers of users, it does have the disadvantage of taking a long time to select each user. An easy solution to this problem is to allow users to search directly on the page using AJAX.

AJAX searching can be achieved by putting an entire search form right on the page and submitting it over AJAX, or it can be done by building a combo box-style search box. As the user types, AJAX search requests are sent and the results are used to build a dynamic pull-down list below the text-input box. An example of this is shown in Figure 4-1. If this is the only AJAX you're adding to an application, it will require the following actions:
  • Addition of an AJAX communication layer (usually provided by a library)

  • Additional JavaScript and HTML code for building the drop-down elements

  • New entry point to the application for getting unformatted search results

  • Testing of the new feature

  • Testing of the fallback to the old pop-up in non-AJAX browsers


AJAX-driven user selection

As you can see from the list of actions, there are no wildly different additions to the development cycle. You just need some extra time to add in this AJAX feature. The biggest changes caused by this addition are the new entry point for your AJAX search and any debugging hassles caused by it.

Changes Caused by Creating an AJAX-Driven Application

AJAX can be useful when used as an enhancement tool that is targeted at specific tasks that are slow or hard to do within the normal Web environment. However, AJAX is most powerful when you rethink how you build Web sites and design it into the application from the ground up. This allows you to move from a design built on full-page reloads to an event-driven application where events drive small areas of the application to update. Requiring AJAX for your application does limit which browsers you can support, but in many circumstances, the tradeoff is worth it, because it allows you to create applications that are not possible in a normal Web application model.

Creating an AJAX-driven application is a much larger shift than just sprucing up a site with AJAX. One of the most invasive changes you'll find is the removal of the page concept from the application. In a normal Web application, you would go to the /list/users.php URL, and that page would generate HTML to show a table of users. In our AJAX-driven application, an event will be fired (maybe by clicking a button) that will cause JavaScript to load user data using AJAX and then update an existing table using the JavaScript Document Object Model (DOM). Depending on your design, the /list/users.php page on the server might still exist. However, it would now just output the data needed, not an entire page. The main shift here is moving the logic that puts together the different data sources from your server to the client.

This shift will have a number of effects on your server-side development efforts. Because you will no longer be generating HTML at each URL, you'll find yourself in a model where you're building an API instead of a bunch of pages. This will have a large effect on most development frameworks, because the front controller will lose command over the actual HTML-generation aspects of the page and instead will focus on data aggregation and security. By the same token, security handling will move to different parts of the application. Because much of the HTML generation will now be handled on the client, you will no longer rely on it to filter out records the user shouldn't be able to access. Although most people won't edit the JavaScript code that drives your application, there is nothing stopping them from doing so; therefore, security checks and filters that are implemented in JavaScript could be removed easily by a determined foe.

The heavy focus on JavaScript will also be a change for many developers; reusable widgets that were created in a template language before will now need to be moved to JavaScript constructs. You'll also find it harder to stay away from JavaScript functions that have inconsistencies between browsers. This is especially true of JavaScript events, because they are at the heart of any event-driven application. Events do work well in all new browsers, but there are some differences, especially with change events that make them unreliable when used by themselves. In most cases, a bit of research will find workarounds to these cross-browser problems, but this aspect will likely be an annoyance during development and will surely increase the amount of testing that needs to be done.

Testing needs will increase as you make a more complex user interface (UI), but this process is relatively straightforward; if you already implement a detailed testing plan, this won't be much of a change. The biggest difference is that more interaction is needed with the elements on the page to test their interactivity. The JavaScript-to-server communication layer also becomes a new point of testing. You will need to make sure that your application handles communication errors properly because you'll be handling them now (instead of leaving them to the browser). The actual testing of the server side will also be a change for developers who are not currently using some type of unit testing processes. In an AJAX application, it is important to be able to test the server separately from the client because this greatly reduces complexity during the debugging process.

An AJAX-Driven Application Use Case: Mp3act
Mp3act is an open source music management system; it allows you to search, browse, and stream music stored on a server. Although it's not as popular as client-side music management systems, such as iTunes, server-side music management does have a thriving niche, and there are many implementations of this same process using a normal Web development model.

Mp3act fully removes the page concept from the application; the application is divided into two independent sections. One section is the navigation bar, which contains links between major aspects of the application, such as search and playlists. The other section is the content area that is updated as needed, either from links in the navigation area or from an action performed inside the content.

Mp3act provides feedback while waiting for data to load so that the user knows something is happening. (This feedback is useful because the Web browser's loading throbber never moves.) This status message is shown in Figure 4-2; this consistent messaging is used throughout the application and is one of the interface touches that any good AJAX-driven application needs.

Mp3actloading status shown switching to Playlists view

Mp3act performs two major types of AJAX actions: loading new application sections where large amounts of HTML are replaced, and dynamically updating tables. Figure 4-3 shows a table being dynamically updated; the plus button (on the right) is clicked, which adds the songs in the table to the playlist. The new songs are highlighted in green, showing the user that an action has been performed. (The highlight fades away in a couple seconds.) Items can also be deleted from the playlist and can be reordered using the arrow buttons. (Similar highlights with fade affects are used throughout the application.)

Mp3actadding an album to a playlist

When compared to a similar application developed without AJAX, Mp3act requires a large number of development changes, such as the following:

  • Addition of an AJAX communication layer

  • Creation of a JavaScript playlist widget

  • Visual-effect JavaScript code for fading highlights and loading messages

  • Creation of a data-driven API on the server

  • Increased testing of rich UI features, such as reordering

  • Cross-browser testing

  • Loss of support on browsers that do not support AJAX

Although AJAX causes development changes, Mp3act's overall development timescale isn't necessarily longer than a non-AJAX version would require. Because Mp3act was conceived as an AJAX application from the start of its development and because its developers never planned for it to support non-AJAX capable browsers, its development process can skip normal Web-development work, such as building forms to reorder lists.

The one area that might still be overly time-consuming is the processing of developing good cross-browser support. In fact, in the original Mp3act release, Internet Explorer (IE) didn't work at all. Resolving these cross-browser issues, which involved items from XMLHttpRequest differences to various ways to create visual effects, could have been easily accomplished by using a library. However, some cross-browser issues, such as CSS layout, have no easy solution. Even if the overall development time is longer than for a non-AJAX version, the end result a usable, attractive application that works the way a user expectsseems well worth it.

Read more...!

Monday, August 20, 2007

How to Decide on a Request Type

There are two main request types used in AJAX communication: POST and GET. These are the identical choices you have with forms in HTML, and the same rules apply. GET requests can be used when the URL performs no action. (For example, a URL such as index.php?section=main that displays the main section of a Web site is a good use for GET.) A URL such as index.php?action=delete&id=1 should never be used with GET.


The main reasons for this go back to the HTTP standard, which suggests that GET requests should not perform permanent actions and are not allowed to be cached by proxies and other infrastructure. Some Web accelerator products also prefetch GET URLs from pages, which causes huge problems when they hit a URL that deletes a record. GET requests also have the drawback of potentially limiting the amount of data that the client can send to the server; current HTTP specs don't limit size, but many servers and browsers limit the size to around 2,000 characters. POST requests aren't allowed to be cached by proxies or prefetched, and thus are much safer than GET requests.

In most AJAX setups, especially any using an RPC approach, only POST requests should be used. While some of the requests might be just to load new data, it's generally too hard to keep track of which type of request is which. In a document-centric approach, GET requests might be useful, especially when you're using AJAX to load in chunks of static HTML or XML content. If you go with an approach like that, you'll want to use a good naming convention to help keep track of which type of request is needed for each URL.

Read more...!

Friday, August 17, 2007

Remote Scripting

Remote scripting is a technique in which you make a request to the server that directly maps to a function on the server. In most environments, this is usually referred to as a Remote Procedure Call (RPC). Remote scripting approaches differ from document-centric ones mainly in how tightly coupled the server side is to the JavaScript client side. The data formats used are of a more generic nature and are designed to move standard data types, such as arrays or strings, and are not application-specific, such as the schema used by our data XML files in the DOM or XSLT examples.

The general RPC pattern is shown in Figure 3-4 and is as follows:


  1. The JavaScript client code serializes the request data

    into a standard format.

  2. The serialized data is sent to the server.

  3. The data is userialized to a native data type.

  4. The data in native format is used to perform an action,

    usually calling a preregistered function or method.

  5. The results of the server action are serialized back to

    the standard format.

  6. The serialized data is returned to the JavaScript

    client.

  7. The JavaScript client unserializes the data to a native

    JavaScript type.

  8. An action is performed on the result.


The RPC pattern

Any approach that follows this basic pattern can be considered an RPC approach. This can be anything from a simple technique that passes plain strings back and forth to something as complex as an entire Simple Object Access Protocol (SOAP) stack.

RPC approaches fit into four main subgroups:

  • Approaches that use plain text or basic serialization

    such as URL encoding

  • Approaches that use standardized XML schemas, such as

    SOAP or XML-RPC

  • Approaches that use custom XML schemas
  • Approaches that send JavaScript or its subset, JavaScript

    Object Notation (JSON)

All these approaches are used in various AJAX implementations
and, in many cases, are even combined; this is especially prevalent with
approaches that generate JavaScript because few server-side languages have the
ability to natively parse it.

Basic RPC
The simplest RPC approaches send plain text between the client and the server. What distinguishes them from document-centric approaches is that they usually call only a single page on the server, and the results come directly from what a function on the server returns. Like any remote scripting approach, there is a server component and a client component. The server component has a list of functions that can be called by the client (for security reasons, any RPC server should allow calls only to preregistered functions), and it manages dispatching client requests to a function and returning its results. The application flow is this: The JavaScript client makes a call using XMLHttpRequest to the server, sending the function to call and a payload. The server calls the requested functions and outputs the results, and the JavaScript client does something useful with the result. The process is shown in Listings 3-7 and 3-8; Listing 3-7 shows the server side written in PHP, and Listing 3-8 shows the client-side HTML and JavaScript.

rpc-basic-plain.php
1  <?php
2 // functions that can be called remotely
3 function rot13($string) {
4 return str_rot13($string);
5 }
6
7 function reverse($string) {
8 return strrev($string);
9 }
10
11 // list of functions that can be called
12 $functionList = array(
13 'rot13',
14 'reverse'
15 );
16
17
18 // function to call
19 $funcToCall = $_REQUEST['function'];
20
21
22 // set the content type
23 header('Content-Type: text/plain');
24

25 // check whether the function is registered
26 if (!in_array($funcToCall,$functionList)) {
27 die('Unable to call'.$funcToCall);
28 }
29
30 // Get the content from the client
31 $payload = "";
32 if (isset($HTTP_RAW_POST_DATA)) {
33 $payload = $HTTP_RAW_POST_DATA;
34 }
35
36 // call a function and output its results
37 echo $funcToCall($payload);
38 ?>
The server-side component of this basic RPC arrangement is as simple as it could be; this works for small pages, but to build a full site, you would want to move to a more feature-rich solution, like one of the toolkits listed in Appendix A, "JavaScript AJAX Libraries." In Listing 3-7, lines 28 define two small string-processing functions; these could just as easily contain calls to a database or code that builds HTML. Lines 1115 provide an array of functions that can be called remotely; this provides security, locking remote access to a small set of functions that expect input from JavaScript. On line 23, we set the Content-type header. text/plain is used because we're merely sending back string data. Lines 2528 show the security check; it uses the $functionList array we built on lines 1215; it also uses $funcToCall, which we read on line 19. The function we're trying to call isn't in this array; we end the script execution using the die command. The script ends by reading in the POST data that was sent from the form and then calling the requested function with that data as its only parameter.

The HTML page contains an input box and some links to perform some remote actions on the contents of the box. The actions run the two functions registered on the PHP page, reverse and rot13, against the content in the input box. The reverse function returns a string in reverse order, whereas rot13 replaces each character with the one that is 13 characters ahead of it in the alphabet. Now that we have a server to call, we need to build our client HTML and JavaScript page. We will reuse the same HttpClient class, adding in a helper function to allow us to make remote function calls. An HTML page that makes RPCs to the PHP script we built in Listing 3-7 is shown in Listing 3-8.

rpc-basic-plan.html
1  <html>
2 <head>
3 <title>Basic RPC Example, No data serialization</title>
4 <script type="text/javascript" src="HttpClient.js"></script>
5 <script type="text/javascript">
6 var serverUrl = 'rpc-basic-plain.php';
7 function remoteCall(func,payload,callback) {
8 var client = new HttpClient();
9 client.isAsync = true;
10 client.requestType = 'POST';
11 client.callback = callback;
12 client.makeRequest(serverUrl+'?function='+escape(func),
13 payload,'text/plain');
14 }
15
16 function reverseString() {
17 remoteCall('reverse',document.getElementById('string').value,
18 function(result) {
19 document.getElementById('string').value = result;
20 }
21 );
22 }
23
24 function rot13String() {
25 remoteCall('rot13',document.getElementById('string').value,
26 function(result) {
27 document.getElementById('string').value = result;
28 }
29 );
30 }
31 </script>
32 </head>
33 <body>
34 <label for="string">Source String:</label> <input id="string">
35 <ul>
36 <li><a href="javascript:reverseString()">Reverse String</a></li>
37 <li><a href="javascript:rot13String()">ROT 13 String</a></li>
38 </ul>
39
40 <div style="position: absolute; width:100px; height:20px;
41 top: 5px; right: 5px; display:none"
42 id="HttpClientStatus">Loading ...</div>
43
44 </body>
45 </html>
Like the earlier example pages, this one is broken into two main sections. The JavaScript code is at the top, followed by the HTML interface that calls it. Lines 714 contain the remoteCall function, which creates a new HttpClient. It sets up this new client to make an asynchronous POST request, and it uses the callback parameter as the callback function for the request. The function finishes by sending the request to the server. Each request is made to the same PHP page; we just change the query string, setting function to the PHP function we want to call. A new HttpClient instance is created for each request. If you didn't do this, you would need to add in extra logic to keep a new request from being made before an earlier one had finished, because each HttpClient instance can have only a single outstanding request.

Lines 1630 provide the two helper functions that initiate the remote function calls. The first is for reverse, and the second is for rot13. The two functions are nearly identical; they both call remoteCall, passing in the remote function to run, the value of the input box as the payload, and a callback to handle the results. The callback functions get the result from the PHP server and set the value of the input box to it.

The rest of the page is the basic UI. Line 34 contains the input box we're reading from to make remote calls; the input box also gets updated with the results of the calls. Lines 3637 contain links to run the JavaScript functions that make the remote calls. The script ends with a DIV (lines 4042), which is shown while we wait for the server to respond. Figure 3-5 illustrates example output showing a reversed string.

Using a basic RPC example to reverse a string

A simple RPC system is quick to build and has little overhead, but it lacks enough functionality that it's not usually used in larger projects. Normally, you want the ability to pass multiple arguments to the functions on the server side and an easy way to get back something besides a string on the client.

Mimicking a Form POST
Another option for performing AJAX-based RPC is to mimic a form POST. This entails URL encoding the data. URL encoding is the format used in the query string of a GET request; key value pairs are separated by an ampersand, and a basic example is shown here:
ajax=asynchronous+javascript+and+xml&hello=world
As you can see in this example, spaces are encoded as + characters. In addition, =, &, and other non-ASCII characters are escaped as hexadecimal entities. Knowing the actual details of the encoding isn't that important because JavaScript contains the encodeURIComponent() to handle the encoding of each key and value, and PHP will automatically handle the decoding for you. URL encoding can be used to perform basic RPC, allowing multiple variables to be passed, or to submit a form over AJAX. The form submission method is especially useful because you can easily fall back to normal form submission for users who don't have JavaScript enabled.

The example of this takes an HTML form and uses a drop-down element to decide how it's submitted. One mode does a submission to a slightly modified version of Listing 3-8; another submits the form using AJAX, and one mode does a normal form submission. The AJAX and normal form submission submit to the same page, showing you how you can detect an AJAX form submission. Because the AJAX code mimics a normal form submission, the server side can treat the data sent from either input method identically. You'll generally create different output from an AJAX form submission because you need to return only that content that needs to be updated (instead of generating an entire page). The fake form submission example is shown in Listing 3-9, starting with the HTML page and finishing with the two back ends.

Ajax-form.html
1  <html>
2 <head>
3 <title>Basic RPC Example, No data serialization</title>
4 <script type="text/javascript" src="HttpClient.js"></script>
5 <script type="text/javascript">
6 function handleForm(form) {
7 var serverUrl = '';
8 switch(document.getElementById('formAction').value) {
9 case 'normal':
10 return true;
11 break;
12 case 'ajax':
13 serverUrl = 'Ajax-form.php';
14 break;
15 case 'rpc':
16 serverUrl = 'Rpc-basic-urlencoded.php';
17 break;
18 }
19
20 var client = new HttpClient();
21 client.isAsync = true;
22 client.requestType = 'POST';
23
24 // urlencode the payload
25 payload = "ajax=true";
26 for(var i =0; i < form.elements.length; i++) {
27 if (form.elements[i].name) {
28 if (payload != "") {
29 payload += "&";
30 }
31 payload += encodeURIComponent(form.elements[i].name)
32 + '=' + encodeURIComponent(form.elements[i].value);
33 }
34 }
35
36 client.callback = function(result) {
37 document.getElementById('target').innerHTML = result;
38 };
39
40 client.makeRequest(serverUrl,payload,
41 'application/x-www-form-urlencoded');
42 return false;
43 }
44
45 </script>
46 </head>
47 <body>
48 <form action="Ajax-form.php" method="POST"
49 onsubmit="return handleForm(this)">
50 <p><label>Source String:</label>
51 <input name="payload" id="string"></p>
52
53 <p>

54 <label>Submit As:</label>
55 <select id="formAction">
56 <option value='normal'>Normal Form</option>
57 <option value='ajax'>AJAX Form</option>
58 <option value='rpc'>RPC Form</option>
59 </select>
60 </p>
61
62 <p>
63 <select name='function'>
64 <option value="reverse">Reverse String</option>
64 <option value="rot13">ROT 13 String</option>
65 </select>
66 </p>
67
68 <p><input type="submit" value="Submit Form"></p>
69 </form>
70
71 <div id="target"></div>
72
73 <div style="position: absolute; width:100px; height:20px;
74 top: 5px; right: 5px; display:none"
75 id="HttpClientStatus">Loading ...</div>
76
77 </body>
78 </html>
Listing 3-9 makes a form submission perform different actions; this basic setup leads us to a different layout in our JavaScript code than most of the RPC examples. Instead of having a number of smaller helper functions, we end up with a large form handler function that performs many of the same actions, no matter how we submitted the form. In its definition, this function handle Form, which starts on line 6 and continues to line 43, expects the form to be submitted as a parameter. Lines 718 decide how we're going to submit the form. To do this, we create a switch statement around the value of a select element. If we do a normal form submission, we return true, which allows normal form submission to take place. For AJAX or RPC form submission, we set the URL to submit content, too. The rest of the logic is the same because the data is formatted the samewhether we're treating it as a normal form POST or a URL-encoded RPC submission.

Lines 2023 set up an HttpClient instance to make an asynchronous POST submission; HttpClient is the XMLHttpRequest we built earlier and included on line 4. After that, we prepare a payload to send as the POST body. This is done on lines 2634. We loop over each element in the form, and if name is set on it, we add it to the form as the string name=value. Both name and value are escaped using encodeURIComponent, with each form element's value being separated from the next by an ampersand (&). Then a callback handler (lines 3638) is created to perform an action on the results of our remote calls; in this case, it just updates the contents of a DIV using innerHTML. The form handler finishes by making a remote request and returning false. When making a request (lines 4041), it's important to include the correct Content-type, because $_POST will be automatically populated in PHP only when the content-type is application/x-www-form-urlencoded. The final action of returning false is also important. Without it, the form would submit over our HttpClient and then as a normal form.

The rest of the file creates a basic user interface: an HTML form, an output target, and an element to show while we're waiting for the server to respond. Lines 4849 define the form, and the action attribute sets the page that will handle normal form submission requests. onsubmit ties our form-handling function to this form, and the value from this function is returned, allowing it to cancel the normal form submission. Line 51 creates the source string; this will be the payload sent to our RPC functions when doing an RPC submission. Lines 5559 define the select element that lets us select how the form will be submitted; the value of each option matches up with the switch statement in the handler function. Lines 6365 define a select element that lets us pick an RPC function to call on the string payload, and the values of these options match functions registered on the PHP RPC page. The page finishes with a submit button (line 68), a DIV with an ID of target that is used to show the output of our calls (line 71), and a status DIV (lines 7375).

The output from ajax-form.html can be sent to one of two pages: rpc-basic-urlencoded.php or ajax-form.php. Each page interacts with the data in the same way because our encoding works in the form submission handler, which makes each POST request look like a normal form submission. ajax-form.php is shown in Listing 3-10; it's a simple page that checks if this is an AJAX submission or a normal submission and then shows the value of $_POST using var_dump. The check for an AJAX submission is done by looking for the ajax element in the $_POST being set. Unless you set a marker like this, there is no way to tell that XMLHttpRequest was used instead of a normal form submission.

ajax-form.php
<pre>
<?php

if (isset($_POST['ajax'])) {
echo "AJAX Form submission\n";
}
else {
echo "Standard Form submission\n";
}
var_dump($_POST);
?>
</pre>

URL-Encoded AJAX RPC
The RPC handler, rpc-basic-urlencoded.php, is the same code as rpc-basic-plain.php,
except for a change to how the payload is read. Lines 3134 of rpc-basic-plain.php
are replaced with the code shown next. This code sets $payload with the
value of the payload index in $_POST:
31  $payload = "";
32 if (isset($_POST)) {
33 $payload = $_POST['payload'];
34 }
You can experiment with the example by loading Listing 3-9. Figure 3-6 shows what the listing's output should look like. Notice how easy it is to move between a normal form submission and an AJAX form submission. AJAX form submissions can be used with document-based approaches as well as with remote scripting approaches.

URL-encoded AJAX RPC

Adding some basic data encoding to our AJAX requests adds a lot of power and flexibility to AJAX-based RPC. It's a great way to send multiple parameters to the server, and it is a great fit for form type data. The same encoding technique can also be used with non-RPC-based approaches when you want to emulate a normal form submission.

SOAP and XML-RPC
SOAP and XML-RPC are standardized XML protocols for doing remote requests. Many people who have used them before will wonder why they are not used more often. There are a variety of reasons for this, but the mains ones are as follows:

  • The standards are complex and require large client

    libraries.

  • Their encodings are verbose, making for slower

    interaction.

  • The biggest benefits of talking to any Web service are

    negated by the fact that XMLHttpRequest's security model limits you
    to talking to the same server that sent the HTML page.

At present, there are no successful cross-browser implementations in JavaScript for either SOAP or XML-RPC. There don't seem to be technical issues stopping SOAP or XML-RPC from being implemented, except that any implementation will be much larger than the RPC options. SOAP or XML-RPC may become more popular in the future if browsers add native clients, but, so far, any implementations such as the SOAP client in Mozilla have been restricted to signed code or custom browsers built from the Mozilla base.

Custom XML
Some AJAX implementations use various custom XML schemas for transferring data. Although these formats suffer from some of the same data-bloat problems as SOAP or XML-RPC, they are generally much simpler and make a better fit because of this. One advantage of custom XML formats is that a format can be constructed to drive actions on the client side instead of just transferring data. Custom XML schemas based off the current schemas in your application's workflow might also be useful, but these generally fit a document-centric approach better because most of the schemas will be data-specific and don't fit into a generic RPC approach.

The XML example server pages, Rpc-xml.html and Rpc-xml.php, build off the RPC plain-encoded example. One advantage of using this approach over the basic RPC code is that it makes your client code more dynamic, because you can use a set of generic content-replacement functions that are put into action as needed from the server. You can use these generic functions instead of coding lots of custom callbacks. Depending on your needs, you may want to use a different XML schema in each direction, but for this example, we use the same one. It's a basic schema that tells which function to call and the parameters to pass to the function. An example of the schema is shown in Listing 3-11.

Example Call XML
<call function="reverse">
<param>Test</param>
</call>
On the JavaScript side, we can use the DOM to read the XML, turning it into a string that can be run through the eval function to make the actual call. This works well from a security standpoint because the browser security sandbox keeps any remote requests on the same server, so you can trust any new content just as much as you trusted the original page load. On the PHP side, you don't have that same level of trust, because a request can come from anywhere on the Internet. Instead, you'll want to compare the function to call against a white list and use a method like call_user_func instead of eval. XML processing is also slightly harder than in JavaScript, because versions older than 5.0 don't have DOM support by default. An easy way to support PHP 4 and 5 is to use a library from PEAR called XML_Serializer. XML_Serializer has the capability to take an XML file and turn it into a native PHP array. Most recent PHP installs come with the PEAR package manager, allowing you to install the library by running pear install XML_Serializer. (Detailed installation instructions are available at http://pear.php.net.) An example using XML to build an AJAX RPC system is shown in Listing 3-12.

Rpc-xml.html
1  <html>
2 <head>
3 <title>XML RPC Example</title>
4 <script type="text/javascript" src="HttpClient.js"></script>
5 <script type="text/javascript">
6 var serverUrl = 'Rpc-xml.php';
7 function remoteCall(func,payload) {
8 var client = new HttpClient();
9 client.isAsync = true;
10 client.requestType = 'POST';
11 client.callback = function(result) {
12 var call = '';
13 var callNode = client.xmlhttp.responseXML.firstChild;
14 call += callNode.getAttribute('function')+'(';
15 var params = callNode.getElementsByTagName('param');
16
17 for(var i = 0; i < params.length; i++) {
18 call += "'"+escape(params[i].firstChild.nodeValue)+"',";
19 }
20
21 call = call.substring(0,call.length-1)+')';
22
23 eval(call);
24 }
25
26 payload = '<call function="'+escape(func)+'"><param>'+
27 escape(payload)+'</param></call>';
28
29 client.makeRequest(serverUrl,payload,'text/xml');
30 }
31
32 function replace(id,value) {
33 document.getElementById(id).innerHTML = value;
34 }
35

36 function append(id,value) {
37 document.getElementById(id).innerHTML += value;
38 }
39
40 function remote(func) {
41 remoteCall(func,document.getElementById('string').value);
42 }
43
44 </script>
45 </head>
46 body>
47 <label for="string">Source String:</label> <input id="string">
48 <ul>
49 <li><a href="javascript:remote('reverse')">Reverse String</a></li>
50 <li><a href="javascript:remote('rot13')">ROT 13 String</a></li>
50 </ul>
51
52 Output:<div id="target"></div>
53 <div style="position: absolute; width:100px; height:20px;
54 top: 5px; right: 5px; display:none"
55 id="HttpClientStatus">Loading ...</div>
56
57 </body>
58 </html>
Listing 3-12 follows the normal pattern of a JavaScript section at the top and then a small UI to interact with it below. The JavaScript starts on line 4 by including the standard XMLHttpRequest wrapper. After that, we define the remoteCall method. This method is based on remoteCall in rpc-basic-plain.html; the biggest different is that instead of a callback being passed in, it is built from the XML. Lines 810 create an HttpClient instance and set it up for an asynchronous POST request. Lines 1124 build the callback handler that handles this result; this code uses the DOM representation of the resulting XML file to perform an action.

The basic process is to build a string and evaluate it. This process starts on line 13 by grabbing the root node of the XML document (the call tag) and putting it into callNode. Next, we append the function name and an "(" to the call string. After that, we use getElementsByTagName() (line 15) to get an array of all the param tags. We loop through these tags, appending each one inside single quotes to the string (lines 1719). Each value is also run through the escape function in case it contains a single quote or another character that would cause our eval() to fail. At the end of this process, we will get a string like "functionName('param1', 'param2',". To finish up the process of building our call string, we remove the extra "," from the end and append the closing ")" on line 21. Finally, the string is run through eval() on line 23, calling the function.

Lines 2627 prepare the XML payload to be sent to the server. Because this is just a simple example with one parameter, it is just a matter of escaping the input and putting it together with the XML tags using string concatenation. Line 26 finishes remoteCall() by making the actual server request.

Next, the file contains a couple of generic callback functions and a helper function for making the remote calls. Lines 3234 contain a generic function that replaces the content of an HTML element using innerHTML. This method takes two parameters: the ID of the element and the content to use for innerHTML. Lines 3638 contain an append() function that follows the same pattern as the replace() function on lines 3234; the only difference is that append() appends to innerHTML. The remote() function on lines 4042 grabs the input value from the string input box and calls remoteCall() with it, passing it the function in the func variable.

The rest of the file (lines 4657) contains the basic HTML UI. Line 47 contains our source input box, and lines 4850 contain a list of action links that call the JavaScript remote() function. Line 52 contains our target DIV, which can be used by the append and replace functions. The file is finished by a status DIV on lines 5355. The back end for this page is shown in Listing 3-13.

Rpc-xml.php
1  <?php
2 require_once 'XML/Unserializer.php';
3
4 // functions that can be called remotely
5 function rot13($string) {
6 $xml = '
7 <call function="replace">
8 <param>target</param>
9 <param>'.str_rot13($string).'</param>
10 </call>
11 ';
12 return $xml;
13 }
14
15 function reverse($string) {
16 $xml = '
17 <call function="append">

18 <param>target</param>
19 <param>'.strrev($string).'</param>
20 </call>
21 ';
22 return $xml;
23 }
24
25
26 // list of functions that can be called
27 $functionList = array(
28 'rot13',
29 'reverse'
30 );
31
32
33 // set the content type
34 header('Content-Type: text/xml');
35
36
37 // Get the content from the client
38 if (isset($HTTP_RAW_POST_DATA)) {
39 $xml = $HTTP_RAW_POST_DATA;
40
41 $unserializer = new XML_Unserializer(
42 array('returnResult'=>true,'parseAttributes'=>true));
43 $data = $unserializer->unserialize($xml);
44
45 // function to call
46 $funcToCall = $data['function'];
47
48 // params to function
49 $params = array();
50 if (isset($data['param'])) {
51 if (is_array($data['param'])) {
52 $params = $data['param'];
53 }
54 else {
55 $params = array($data['param']);
56 }
57 }
58 }
59
60 // check whether the function is registered
61 if (!in_array($funcToCall,$functionList)) {
62 die('Unable to call'.$funcToCall);
63 }
64
65 // call a function and output its results
66 echo call_user_func_array($funcToCall,$params);
67 ?>
The first section of the file contains the same small function wrappers around basic PHP string handling functions as the plain example. The difference is that returned small chunks of XML define what to do with the results instead of just sending the results back. The rot13 function on lines 513 returns XML to replace the content of the element with an ID of target with a rot13 of the input string. The reverse() function on lines 1523 returns XML to append to the contents of the target element with the reverse output of the input string. These two functions are then added to our function white list on lines 2730.

The latter half of the file (lines 3366) takes an incoming request and prepares the results. Line 34 sets the content-type; if it's not set to text/xml, then responseXML will never be populated on the client without this header being set. Lines 3758 parse the XML, getting the function to call and building an array of parameters to call it with. Lines 4142 create a new XML_Unserializer instance. Options are set to parse XML attributes (line 42) and to return the parsed data form unserialized instead of using an extra method call to get it. Line 43 parses the actual XML and sets its output array to the $data variable. Line 46 uses this array to get the function we're calling, and then lines 4957 grab the array of parameters. We first check whether the param index is set (line 50), allowing us to call functions without input. If it is set, we check whether it's an array (line 51). If it is, we just set that to $params; if it's not, we wrap it in an array as we set it to $params. This is done because XML_Serializer makes $data['param'] an array when multiple param tags exist, but if just one exists, XML_Serializer makes $data['param'] index string. Lines 6063 do a basic security check, canceling script execution if the function isn't in our white list of AJAX-callable functions. Finally on line 66, we use call_user_func_array to call the function and echo its results to the client. This example is shown in Figure 3-7.

XML-based AJAX RPC

Using XML to move the data in our AJAX RPC system, we can build a system that can transfer any type of data. Using the concepts of standard XML-based RPC systems, we can encode any type of data and get the flexibility needed to build complex applications. If we optimize the schema for the server-side language that is being used, we can also limit the overhead created by the XML tags needed to describe the data. XML is used in many AJAX RPC libraries and provides everything you need to make a complete RPC solution.

JavaScript and JSON
Generating JavaScript and sending it to the client where it is run through eval() is a popular way to move data in object implementations. This process is popular because JavaScript has compact notations available for data types, such as arrays, and it allows for very flexible operations. Some AJAX frameworks use this ability to generate new client code from the server as needed, allowing the framework to provide a more centralized view of the development instead of the normal client/server dichotomy.

A mixed XML/JavaScript example could easily be built for the RPC XML in Listings 3-12 and 3-13. On the client side, you would send XML to the server, as is the case with the current code, but for the results, the server would return JavaScript and the client would eval it directly. To accomplish this, we need to make some small edits to Listings 3-12 and 3-13. For Listing 3-12, we need to replace lines 1224 with a simple three-line callback that will run eval on the results from the server. This new callback is shown in Listing 3-14. The server URL on line 6 is updated to point to a new PHP script, which will be an edited version of Listing 3-13. This update is shown here:

var serverUrl = 'rpc-xml-javascript.php';

On the server side, lines 423 of Listing 3-13 are removed and replaced with the eight lines shown in Listing 3-15. The content type on line 34 of Listing 3-13 was also changed to text/plain.

Changes to Make Rpc-xml-javascript.html
12 client.callback = function(result) {
13 eval(result);
14 }
More changes to Make Rpc-xml-javascript.html
4  // functions that can be called remotely
5 function rot13($string) {
6 return "replace('target','".addslashes(str_rot13($string))."')";
7 }
8
9 function reverse($string) {
10 return "append('target','".addslashes(strrev($string))."')";
11 }
These small changes give you a simpler code base with which to work, and their use entails sending a much smaller amount of data back to the client. It also allows for simpler, more flexible coding because you can send back any JavaScript instead of only what your XML schema allows.

Taking this approach one step further and sending JavaScript in both directions would be nice because of the data savings, but that's a much harder task to do for a couple reasons. Most server languages don't contain a JavaScript interpreter, so they can't evaluate JavaScript code, but even if they did, you wouldn't want to allow arbitrary client-created code to run on your server. Running code from the client on the server would be a huge security problem. The solution to both of these problems is a subset of JavaScript called JSON, which is the literal syntax for JavaScript objects. It can be run by eval() on the JavaScript side. This allows for any JavaScript data types to be transferred, and it's much faster than an XML-based solution because the compact encoding allows for a much smaller amount of data to be transferred. On the server side, JSON is simple enough for a small parser to be built to serialize native data types into JSON and to create native data types from JSON.

While JSON is powerful and supports all JavaScript data types, it also has the drawback of being a more complex solution. A library is needed on the server to handle moving to and from the JSON strings, and a library is needed on the JavaScript side to create JSON stringsalthough eval() can be used to turn JSON into JavaScript objects. This makes a JSON example more complex than the examples shown in the other sections, which means if you want to use JSON, you'll need to find some external libraries to do the actual parsing. Appendix A contains a list of libraries that provide JSON processing. In Chapter 9, "Libraries Used in Part II: HTML_AJAX," a complete JSON RPC library for PHP HTML_AJAX is shown.

Read more...!

Wednesday, August 15, 2007

Document-Centric Approaches

A document-centric approach to AJAX simply means that your main interaction with the server is pulling down pages of content. This doesn't mean that the pages aren't dynamically generated, but it does mean you're pulling down the content in a ready-to-use or parse format. The simplest use case is to download a chunk of HTML from your server and insert it into the page using innerHTML.

The biggest differentiator between document-centric approaches and remote scripting is how the design relates with the server. Remote scripting-style AJAX is tightly coupled to the server and gives you a direct interface to the server-side code, using a standardized system to transfer the call and request between the client and the server. Document-centric approaches are loosely tied to the server; the only requirement is that the data be in the usable format, allowing you to generate plain text, HTML, or XML in whatever manner you want. One advantage that document-centric approaches have is that they are highly scalable because more of the work happens on the client. Document-centric approaches are not a magic bullet for scalability or performance, because dynamically generated content can have many bottlenecks; however, they do closely resemble current Web models, which allow the same optimization strategies to be used.

Adding New HTML Content to a Page with AJAX
One of the basic actions performed on every Web page is the displaying of new content when a user clicks a link. In many circumstances, this works fine, but when you want to keep the original content, you have a problem. HTML offers two solutions to the problem: frames and IFrames. Both allow multiple pages to be embedded into a single page. The problem with frame-based solutions is that you're still stuck loading entire pages; that being said, you can easily use frame-based solutions to add a new row to a table or provide a status message. AJAX offers an easy way out: load the HTML directly using XMLHttpRequest and add it to your page by replacing the content of a DIV using innerHTML. An example of using the HttpClient.js XMLHttpRequest wrapper we built earlier (Listing 2-3) is shown in Listing 3-1. The pages that are being loaded, content1.html and content2.html, are also shown; they can be any fragment of HTML or other text you want to load into the DIV.

Adding HTML Content.html

Line 4 includes the XMLHttpRequest wrapper, HttpClient, which is instantiated on line 7. Because we want to make an asynchronous request, we turn that mode on at line 8 and then create our callback function (lines 1012). This callback will take the result from the load and set the innerHTML of the target element to it. Lines 1416 create a JavaScript function that we can call from HTML links. This function, replaceContent, takes a single parameter, page, which is the URL we want to load. Line 15 makes the remote request. The second parameter is null because we have no POST data that we want to send as a payload.

The rest of the example is a list of test calls to replaceContent and the user interface (UI) elements used by the JavaScript code. Lines 2223 load Content1.html, and lines 2425 load Content2.html. Lines 2830 create a DIV element with an ID of HttpClientStatus. Whenever you are using HttpClient.js, you need to provide an element with this ID. The element should be hidden by default; this is accomplished by setting display:none in the style attribute. When the XMLHttpRequest object downloads content from the server, the HttpClientStatus element will be shown by setting the style.display attribute to block. Lines 3233 provide a DIV with an ID of target; this is where the downloaded content is displayed. The results of Listing 3-1 loading content1.html are shown in Figure 3-1.

content1.html

<p>I'm an HTML fragment. Any <b>html</b> can be used.</p>

content2.html

I'm some other content. In this case I don't contain anything but text.


Using AJAX to perform basic content replacement

In Listing 3-1, we took a normal HTML document and used AJAX to load in new content at will. This basic technique can be accomplished through the use of frames, but AJAX gives you a lot more flexibility. We can divide the page in any way we choose, dynamically updating something as small as a single word or as large as the majority of the page. Dynamically updating a page with content is powerful, and it fits well into any server-side development model, because you're simply increasing the number of pages you generate while decreasing their size and scope.

Consuming XML Using DOM
Another popular way to load new data is to use XML. XML is useful when you want abstraction between the code on the server that produces the data and the JavaScript client code that uses it. This allows you to change out the back end without affecting the front end. It also allows you to expose data for other clients through the same generic API. The XML Document Object Model (DOM) is similar to the HTML one that we've used in other examples.

An easy way to visualize the DOM is to picture a tree of objects, one for each XML element on the page. For example, the following sample XML document will make DOM with four nodes: the root Document node, an Element node for the rootTag, and an Element node for the childTag. The childTag has a Text node containing Some Text.

<rootTag>
<childTag>Some Text</childTag>
</rootTag>


A visual representation of this tree is shown here:

Document
Element (rootTag)
Element (childTag)
Text


To turn this DOM model into HTML, we will need to use a couple basic methods and properties:
• getElementsByTagName(tagName). Gets an array of tags of the specified name as a result
• getAttribute(attributeName). Gives you the value of one of the tag's attributes
• firstChild(). Returns the first child node of any node

The other step we need to take is to get the XML document result from XMLHttpRequest instead of the plain text. To do this, set the Content-type of the downloaded page to text/xml. This causes the responseXML property of XMLHttpRequest to create a DOM document from the contents.

In the following examples, we take an XML list of resource links about a subject and turn them into a simple HTML list. The two test lists contain PHP resource links and a list of book sellers. Listing 3-2 contains the list of PHP resources, whereas Listing 3-3 contains the list of sellers. These files are used by Listing 3-4, which uses the DOM to update the current page and build the list.
PhpResources.xml

BookSellers.xml

DOMExample.html

The JavaScript HttpClient class (from Chapter 2, "Getting Started") gives us cross-browser XMLHttpRequest support. On line 4, we include the library, and on line 7, we create an instance of the client. Then, on line 8, we set isAsync to true because we want to make an asynchronous request for the XML data file. Next, on lines 1028, we add our callback function; this function takes the downloaded XML document, creates an HTML list, and then shows the list using innerHTML.

On line 11, we grab the XML DOM document from response XML. We have to use the XMLHttpRequest object directly because HttpClient doesn't wrap this. Depending on the complexity of the HTML page, updating a node with innerHTML can be an expensive operation. To keep this to a minimum, we use a variable to hold our HTML content and then update it all at once at the end of the function. On lines 1315, we read the type attribute from the site's tag and use it to make a title for our list. Then on line 17, we get an array of all the site nodes in our XML document, which is looped through on lines 1823. In each iteration of the site's array, we build one list element. This is a pretty straightforward process; the only item of note is the use of firstChild() and nodeValue() to get to the text content of the site tag. These calls are needed because text content exists in its own node in the DOM, and there is no innerHTML attribute to read from the text content and its markup, as is the case with the HTML DOM.

The rest of the page gives you a basic UI for testing. Lines 3032 provide a helper function that requests the download of new XML files. When the download is done, the callback function that builds the output will be called. Lines 3742 contain a list with links to process the sample XML files, and lines 4446 contain a basic loading DIV that is shown while waiting for the XML documents to be downloaded. Finally, we have a target DIV that is used by the callback function as a place to display the generated list. The output of Listing 3-4, showing the PHP resources list, is shown in Figure 3-2.

Updating an HTML page by consuming XML documents using the DOM

When tied with dynamically updates, DOM-based consumption of XML can be an efficient way to dynamically display data on the browser. Using the DOM manipulation function can make for tedious programming, so it isn't usually the best approach for generating a large amount of content from nonstructured data.

Consuming XML Using XSLT
eXtensible Stylesheet Language Transformations (XSLT) is another popular way to take a DOM document and generate new output. The advantage it has over DOM is that the transformation and the data are in an XML file. XSLT has been used by many successful AJAX applications, such as Google Maps, but it does have a number of drawbacks. XSLT browser support is uneven, and even when two browsers, such as Internet Explorer 6 and Firefox 1.0, support the same main features, the application programming interfaces (APIs) for controlling the transformations from JavaScript are completely different. This difference is large enough that you can't just write a simple wrapper like you can for XMLHttpRequest. If you need cross-browser support, you'll need to rely on a library like Sarissa instead. (The Sarissa library is explained in detail in Chapter 8, "Libraries Used in Part II: Sarissa, Scriptaculous.")

XSLT can also be problematic simply due to its complexity. Not only will you need to learn how to write the XSLT style sheets that drive the actual transformation, but you'll also need to learn XPath, which is used to refer to XML nodes inside the style sheet. Because XSLT is a World Wide Web Consortium standard, there are tools and documentation out there to help, but in many cases, the added effort required over a DOM approach isn't worth the effort.

Although the purpose of this book isn't to teach you how to write an XSLT style, I will explain the basics of the one used. Listing 3-5 replaces the JavaScript DOM code in Listing 3-4 with an XSLT transformation. The same XML data files (PhpResources.xml and Booksellers.xml) that are used in the DOM example are used here. The Mozilla XSLT API is used in these examples because it's easier to understand, and all you have to do to make it work in IE is include the Sarissa library.
XSLTExample.html

The first 18 lines cover the basic setup; we include our HttpClient XMLHttpRequest wrapper and the Sarissa XML compatibility library. On line 7, we create an HttpClient instance; this will be used to load both the style sheet and the XML files we're going to transform; on line 8, we set isAsync to true because we will be making only asynchronous requests. On line 9, we create a new xsltProcessor instance; this will be loaded with a style sheet in the setup function (lines 1118) and then used to transform XML files loaded by the displayResources function (lines 2030). On lines 1316, we create a callback to run when the style sheet is loaded. It grabs the new XML DOM from the client (line 14) and then adds it to the xsltProcessor using its importStylesheet method (line 15); this style sheet is shown in Listing 3-6. The setup function completes by making the actual request (line 17) and is run by an onload handler (line 33).

The displayResources function is called by links in the HTML page; it loads new XML files and then transforms them. Lines 2127 add a callback method that processes the downloaded XML document. Line 23 uses the processor we created to generate a new DOM document formatted by the style sheet we imported on line 15. Line 24 clears the target element on the HTML page, and then on lines 2526, we append the content of the transformed DOM document. document.importNode has to be used for this process to work in a cross-browser manner. The rest of the HTML page (lines 3449) has no changes from the DOM example. It's just a basic list of actions to be performed and a target to display the results.

Listing 3-6 finishes up the process; it is run on each XML file to produce HTML that is similar to the DOM example. Line 8 creates the list title using value-of to output the type attribute. Lines 1019 loop over each site tag in the file, outputting a list item with the link inside it. You can see the output of this script showing the PHP resource list in Figure 3-3.
Using XSLT to transform XML documents loaded using AJAX

Resources.xsl

XSLT is an extremely powerful technique for managing AJAX transformation, and because it's supported by most browsers, it's easy to see how it could be paired with AJAX. XSLT's strength lies in its ability to create rules that will work against nonstructured or structured schemas. This trait allows it to easily transform any type of XML document and generate new content to add to the current HTML page. If you are already dealing with XML on the server side, XSLT makes a great choice because there is a good chance you're already familiar with its basics.

Read more...!