Communication between web pages and server applications

Web sites were originally intended as a tool for reviewing academic papers, but have evolved into a platform for applications that people use every day. That is a big change, and to enable that change software developers have created many tools and discovered some patterns along the way. Some tools and patterns  have worked well; and some not well at all. Here I present some findings from making web application over the last 20 years. Lots of learning in that amount of time!

This is part 2 of a set of posts; so this article assumes you can make a web page with HTML and javascript; but have a hard time understanding how that page works with a server, api, and not sure about the http, ssl, or other terms that you have heard along the way.

What are we making?

For this example we want something simple, but has some logic to understand how the pieces work. For this we have a fictitious client named Terry. Terry works at the weather office and needs a web app to track weather data, like temperatures. Terry needs a weekly report for this weather data, and is waving around pile of money to get it done. Let's get this done for Terry. 

How are we making it? 

We are going to use Python to accomplish this task. There are many choices when it comes to web servers and application are built in various languages (Java, PHP, ruby, javascript) but we are going to use just Python to cut down on the complexity of setting up a lot of tools

Servers
First lets understand how the page in your browser interacts with 'servers'. When you type the address in the browser you are requesting content from that address. What does that mean? It's really the same as opening the file on your own computer, and you notice in the address bar the difference; opening the file locally has the address starting with file: and opening the file from a server has the address starting with http: What are those indicating? They are the protocols the browser uses to get the content. Files are local to your machine when you are using your machine, so the 'file' protocol is used to get that. Files served from web servers is content from another machine, so the browser uses the 'http' protocol to get those. This stands for 'hyper text transfer protocol', and it is exactly that; a protocol to transfer hypertext files around. I suppose the 'file' could be renamed "hyper text file protocol" but htfp isn't that descriptive, so file is used.

Whats after the protocol? the name of the domain you are requesting the resource from; this is whatever.com or your favorite site

Start a web server on your local machine, and get the file with your browser from that webserver, instead of your file system. You get to use the http protocol to do this, and you use the domain name of your local machine. This is the 'localhost', but as always that name maps to a IP address, and in the localhost case this IP address is 127.0.0.1. All domain names map to IP addresses. This is the internet magic!

Use apache or nginx or any webserver to do this; here is a handy link to get started with the simple http server in python: https://developer.mozilla.org/en-US/docs/Learn/Common_questions/set_up_a_local_testing_server

Methods
HTTP protocol has a number of methods to interact with webserver to get some content from the server or send some content to it. These are requests and responses, and in http we will cover the most commonly used requests; GET and POST.
GET is what is says; I want to get some content, and POST is I want to send some content. If you type your address in the browser its a GET request, but if you fill out a form in a page and press save; you are probably sending a POST request.

The vast majority of web and application requests are GET requests, and those just fill your browser with HTML text content. Sending data to the server is a different story so lets dive into details of the POST and how to use it to make web applications. Use your html skills to create a form element, and inside that put your input field and submit button. When you POST to localhost, you are sending a request that contains all the info about your browser, and the value of the field you just filled in. How do you handle that with the server?

Flask
The application server we are going to use here is Flask. There are many different ones for many different languages and this isn't a comparison post, but instead an effort to explain the basics and apply some fundamentals to get you on the road to web development nirvana.

http://flask.pocoo.org/docs/1.0/quickstart/#http-methods

Web pattern 1.0

GET the page and POST your data. The server will get the data from the POST, but what happens when you hit refresh on your browser? The same data will POST again. This can be an issue, since the user just wanted to send the data once, so in your boundary object the good practice here is redirecting to a GET request, so the data is saved and you don't have the issue of re-submitting data. This gets us further, but 2 problems now remain.

1. To show everything on the page you have to get all data to do that, plus any changes to the data from the POST call
2. The experience from the user isn't so fun as the browser is constantly getting redirected, causing a 'flicker' effect when dealing with many pages.

So, wouldn't be nice to just leave that page in the browser, and POST the data some other way? This was the impetus for the technology pattern we know as AJAX. AJAX lets you POST using javascript and get the response from the server through the same function. Browsers after the late nineties supported this feature.

Back then you had to get creative. I made a Java applet back in 1998 that was very much an AJAX app. It was only about 60kb and it just had the basic networking to get and post xml text to the server. Another applet was the toolbar in a frame, which switched the pages in the main frame. It was a reporting tool for a server side application. The applet interfaced with the javascript in the browser through a JSObject object from netscape. Getting this Java applet to work in IE on the Mac was quite the trick, but the experience using the page was very slick. About that time the httpRequest object became available in many browsers, and AJAX became a widely accepted pattern; and spawned a new buzzword: web 2.0.

Patterns learned

GET to POST.

• Use Get to initialize a view of html, but don't pass data to the server using GET (unless its a temporal piece of data like a token)

POST redirect to GET. 

• When POSTing data, redirect the users request to a GET request that confirms what they posted and prevents resubmits from the form that aren't necessary.

Web pattern 2.0

With the advances in web page development as a graphical platform, and the underlying AJAX technology, web pages started to feel like desktop applications. Plug-ins like java applets and flash were taking hold, but developers were seeing how the html/css/javascript basic web technologies were maturing into a package we could build applications in.
Around this time XML was the cool new technology. It was a way to interchange data on common http protocols, and easy to use because it was just formatted text. The X in AJAX is for XML, and for a few years we were posting XML and getting XML responses and updating our web page elements with those results.
This was great, but one more improvement was to come. Since we use JavaScript through the browser, and sometimes on the server, the use of XML as a way to interchange data became a bit cumbersome. A JavaScript guru named Douglas Crockford came up with a way to represent all the capabilities that XML has to interchange data, but use JavaScript for that. JSON was born and it's now the better way to send and receive data from the server

Patterns learned

POST Json with AJAX and do so consistently for any form. On the server its much easier to deal with consistent data structures, and with this pattern the client and server have a common type of data structure to use. Why didn't the term become AJAJ? I guess that may be more correct, but let's not get caught up in the pedantic details and move forward.

Security and Performance
SSL, sensitive info

keep the payload light, avoid redirects

Server patterns are next