This is an article in a series of unknown length discussing a tool written in Lua to publish posts on a WordPress blog from a PC. The source code is available in a public fossil repository.

• Part 1 showed how to use XML-RPC and wp.newPost
• Part 2 added file reading to make a minimal working utility.
• Part 3 added amenities and made the utility useful.
• We digressed to talk about OAuth and using it at WordPress.com.
• Part 4 switched to cURL, REST, and OAuth, but can’t yet post.
• Part 5 improved the token handling and user display and still can’t post.
• Part 6 implemented posting and added the ability to set title, tags, and categories from command line options.
• Part 7 added a new utility and created modules common to all our utilities.

In this installment we will implement a minimal web server that we can use to collect the OAuth token credentials directly rather than depending on the user to copy and paste information manually from their web browser to our command line.

Why do we want this?

OAuth is designed to use a brief web interaction where the user signs in to their data provider and authorizes the application’s use of their private data. For best and clearest security, the sign in and authorization steps should be carried out using the system’s default web browser. The final step of that process has the user’s browser fetch a page from an URL that is specified as part of the application’s registration, with URL query parameters that contain the token and related details.

As a simple command line utility, however, the WPCLI tools don’t, can’t, and won’t contain a web browser. But if a simple web server were available, the registered URL could point to the local host and that server could be used to collect the token without further user interaction.

The scenario would go something like this:

1. User asks a WPCLI tool to authenticate with their blog: wppost --authenticate example.wordpress.com
2. The tool launches the user’s default web browser on the WP authentication page.
3. The tool starts its internal web server.
4. User agrees this is what she wants, fills out the form and submits the authorization.
5. Browser fetches http://localhost:8080/authorized?...
6. The tool’s web server captures the token, expiration, and site codes from the fetched URL and returns a reassuring document for the browser to display.
7. The tool stores the token in its configuration file for later.

In order to do this, the primary missing piece is a simple web server that can be depended on to capture the URL and serve up a limited number of page requests.

Note that there is one minor complication: the port number on which we run a server is formally part of the URL known to WP as part of the application registration. But if the port number is already in use on the user’s PC, we would like to use a different port. To mitigate this a little bit, the scenario as shown uses port 8080, which is a well-known port for transient or local web servers. If that port were already in use, the scenario should error out at about step 2.

So we build out the full interaction with the remote site, a browser, and the WPCLI utility’s configuration file, we should start with the web server itself.

Implementation

The web is based on the protocol known as HTTP which is at its core a fairly simple command/response protocol. A lot of convention and extension has happened since it was first proposed in the early 1990s but the bits we need to deal with for this application are still very simple.

A web server opens a socket, binds it to a TCP port (usually port 80, but we’ll use 8080 for now), and listens for a connection from a web client. When a connection arrives, it reads the client’s request, formats a response, returns it, and closes the connection. (I have left out a lot of details that are important for high-performance servers here. This server need not be high performance, and will continue to ignore all of those features.)

Working in Lua, the accepted best way to access TCP ports is to use the LuaSocket module. Since that module is already included in Lua for Windows, that is simple to do. LuaSocket provides the means to create a socket, accept a connection, and transfer data. We still need to use those features to implement HTTP. LuaSocket does provide a basic implementation of an HTTP client, but not a server.

However a little bit of searching turned up an experimental server written in Lua with LuaSocket called Ladle. As is, Ladle is not ideal for our purposes (primarily because it was built as a stand alone script and not a module) but it’s code base will serve as a good starting point. Since it is already licensed under the same license as Lua itself and the WPCLI tools, there are no legal issues with it either.

As I turned it into a module, I stripped off some features, and decided to rename it to “Spoon” because it will spoon up a single web page per call rather than acting as a generic server for a directory tree. The rename also seemed apt because as I worked through it, I found I was changing it fairly substantially. So Spoon is based on Ladle, but is evolving to be quite different from Ladle.

Each individual component of a web page is fetched by a simple transaction. The client (the browser, most likely) sends a request that looks like GET /some/resource/url?query=parameters&here=too HTTP/1.1 followed by a number of additional lines of information about the transaction, and potentially by some content if the method were POST or the much rarer PUT instead of GET. The server uses the URI, including its full path name and any query parameters (and form content for POST) to decide what to return as its response. A full web server will usually map requests to files in its file system. Spoon leaves that detail up to its caller. The server’s reply also contains headers that describe the transaction.

This version of Spoon will ignore all headers from the request (and any POST data) and provide the bare minimum headers in its reply. Future versions could be enhanced to be smarter, but this is sufficient for the immediate need.

The file spoon.lua will contain the module. We’ll use a low-overhead module framework where all of the public functions of the module are declared in a private table that is returned from the module’s file scope; this is compatible with both Lua 5.1 and 5.2.

-- spoon.lua a single serving web server module
-- simplest module framework
local M = {
  _NAME=(...) or "Spoon",
  _DESCRIPTION="A single serving web server.",
  _VERSION="0.001"
}

local socket = require("socket")

The module provides an entry point to setup and run the server. You pass it a table containing fields that describe the server, and most importantly a callback function that will actually generate any non-error responses.

function M.spoon(opt)
    opt = opt or {}

    -- display initial program information
    if M.debug or opt.debug then 
      print(M._NAME..": "..M._DESCRIPTION.."n","Version "..M._VERSION)
    end

    -- if no port is specified, use port 8080
    if opt.port == nil then opt.port = 8080 end

    -- create tcp socket on localhost:port
    local server = assert(socket.bind("*", opt.port))

    -- display message to show web server is running
    if M.debug or opt.verbose then
      print("Running on localhost:" .. opt.port)
    end

    -- handle one or more client requests
    if not opt.loop then
      M.oneClient(server, opt) -- handle a single client request
    else 
      while opt.loop do 
        M.oneClient(server, opt) -- handle a single client request
      end
    end
end

A single request is handled by the oneClient() function. It is provided the bound server socket, and the full options table. It is responsible for listening for a connection, receiving a request, parsing it and calling the appropriate function to generate its reply.

-- TODO: Implement request headers and body to support PUT, POST and DELETE
function M.oneClient(server, opt)
  -- accept a client request
  local client = server:accept()
  -- set timeout - 0.1 minute.
  client:settimeout(6)

  -- receive the Request-Line from client
  -- Note that the balance of the request including all the headers and the POST data is ignored
  local request, err = client:receive('*l')

  -- if there's no error, begin serving content or kill server
  local response
  if not err then
    local method, uri, httpver = request:match"^(%S+)%s(%S+)%s(%S+)"
    if opt[method] then 
      response = opt[method](opt, method, uri, httpver, request)
    else
      response = M.Errorpage(501, '<pre>rn'..request..'rn</pre>rn')
    end
  else
    response = M.Errorpage(500, '<pre>rn'..err..'rn</pre>rn')
  end
  client:send(response)
  client:close()
end

A utility function is provided to wrap a document body with the minimal set of headers to form a full HTTP reply. It is used internally by the Errorpage() function, and is recommended that the GET callback use it too to avoid duplication of code.

It uses the status() function to transform an integer HTTP status code from its numeric value to the text string expected in the response (200 becomes 200 OK, 404 becomes 404 Not Found, and so forth).

It takes an optional argument to override the standard Content-Type header from text/html if required.

function M.response(status, document, mime)
  local resp = {
    "HTTP/1.1 "..M.status(status),
    "Server: Spoon v"..M._VERSION,
    "Content-Type: ".. (mime or "text/html"),
    "Content-Length: "..tostring(#document),
    "",
    document
  }
  return table.concat(resp,"\r\n")
end

Demonstration

With this minimal amount of code, a demonstration can be written that returns a document describing the request for any GET request and 501 Not Implemented for all other methods. Note that we’ve stored the spoon module in the cli folder so it is found by require "cli.spoon".

-- soupspoon, a demo for the spoon web server module
local app = require "pl.app"

-- Add the script's folder to Lua's module paths so that we can easily use
-- modules shared among our scripts. Remember the script's folder in case
-- we want to use it for finding documentation or other resources.
local appdir = app.require_here()

local spoon = require "cli.spoon"
--spoon.debug = true

local function GET(opt, method, uri, httpver, request) 
  return spoon.response(200, [[
<html>
<head>
<title>Spoon full of content</title>
</head>
<body>
<pre>
]]
.. "Method: " .. method .. "\r\n"
.. "URI: " .. uri .. "\r\n"
.. "Raw request:rn" .. request .. "\r\n"
..[[</pre>
<form name="input" action="/boo" method="get">
Sample: <input type="text" name="sample"><br>
More: <input type="text" name="more">
<input type="submit" value="Submit">
</form> 
</body>
</html>
]], "text/html")
end

spoon.spoon{
  port=8080, 
  loop = true,
  verbose=true,
  GET=anymethod,
}

To try the demo, update your working folder to at least this checkin. The soupspoon.lua demo is in the wprest folder, and spoon.lua in the cli folder below that. soupspoon.lua has no command line options or arguments, so just run it.

Once it is running, open your web browser and fetch to see it in action. The document it displays will contain the URI details from the request, and a form you can submit to see that the server can handle query parameters as well.

It would be fairly straightforward to extend the demo to allow the GET() function to fetch different documents based on the URI, or display information based on the query parameters. That is left as an exercise for the next installment.

Repository and Checkins

All of the code supporting this tool is in a public fossil repository. See the discussion at the tools page for how to get started using this repository. This post documents work that was checked in as [3790516583].

---

(Written with StackEdit.)