The page-worker module provides a way to create a permanent, invisible page and access its DOM.

The module exports a constructor function Page, which constructs a new page worker. A page worker may be destroyed, after which its memory is freed, and you must create a new instance to load another page.

You specify the page to load using the contentURL option to the Page() constructor. This can point to a remote file:

pageWorker = require("page-worker").Page({
  contentScript: "console.log(document.body.innerHTML);",
  contentURL: "http://en.wikipedia.org/wiki/Internet"
});

It can also point to an HTML file which you've packaged with your add-on. To do this, save the file in your add-on's data directory and create the URL using the data.url() method of the self module:

pageWorker = require("page-worker").Page({
  contentScript: "console.log(document.body.innerHTML);",
  contentURL: require("self").data.url("myFile.html")
});

You can load a new page by setting the page worker's contentURL property. In this example we fetch the first paragraph of a page from Wikipedia, then the first paragraph of a different page:

var getFirstParagraph = "var paras = document.getElementsByTagName('p');" +
                        "console.log(paras[0].textContent);" +
                        "self.port.emit('loaded');"

pageWorker = require("page-worker").Page({
  contentScript: getFirstParagraph,
  contentURL: "http://en.wikipedia.org/wiki/Chalk"
});

pageWorker.port.on("loaded", function() {
  pageWorker.contentURL = "http://en.wikipedia.org/wiki/Cheese"
});

Scripting Page-Worker Content

To access the page's DOM you need to attach a script to it. In the SDK these scripts are called "content scripts" because they're explicitly used for interacting with web content.

You can specify one or more content scripts to load into the page using the contentScript or contentScriptFile options to the Page() constructor. With contentScript you pass the script as a string, as in the examples above. With contentScriptFile you pass a URL which points to a script saved under your add-on's data directory. You construct the URL using the data.url() method of the self module.

While content scripts can access DOM content, they can't access any of the SDK APIs, so in many cases you'll need to exchange messages between the content script and your main add-on code for a complete solution.

For example, the content script might read some content and send it back to the main add-on, which could store it using the simple-storage API. You can communicate with the script using either the postMessage() API or (preferably, usually) the port API.

For example, this add-on loads a page from Wikipedia, and runs a content script in it to send all the headers back to the main add-on code:

var pageWorkers = require("page-worker");

// This content script sends header titles from the page to the add-on:
var script = "var elements = document.querySelectorAll('h2 > span'); " +
             "for (var i = 0; i < elements.length; i++) { " +
             "  postMessage(elements[i].textContent) " +
             "}";

// Create a page worker that loads Wikipedia:
pageWorkers.Page({
  contentURL: "http://en.wikipedia.org/wiki/Internet",
  contentScript: script,
  contentScriptWhen: "ready",
  onMessage: function(message) {
    console.log(message);
  }
});

For conciseness, this example creates the content script as a string and uses the contentScript property. In your own add-ons, you will probably want to create your content scripts in separate files and pass their URLs using the contentScriptFile property.

To learn much more about content scripts, see the Working with Content Scripts guide.