ExtensionSystem

From Develop

Jump to: navigation, search

Contents

Design Document: Extensions

Summary:

This design document covers adding support for extensions which can enhance, augment, and modify Miro run-time behavior.

It mentions the Extensions API, but that is covered more extensively in ExtensionsAPI.

Note:

This is a design document. It is not a public forum. If you want to talk about extension things, use the Talk:ExtensionSystem page.



Requirements, Use Cases and User Stories

The gist of it is this:

  1. support core extensions which come with Miro
  2. support user-installed extensions in user directories
  3. infrastructure to dynamically discover extensions in platform-appropriate locations
  4. support for extensions that have their own resource files (images, etc) and multiple Python modules/packages
  5. infrastructure and user interface to enable/disable extensions
  6. infrastructure and user interface to install/remove extensions
  7. support updating extensions outside of Miro core release cycle
  8. support user-created extensions
  9. extension registry web-site

Additionally (covered in ExtensionsAPI):

  1. API for extensions to use providing them a stable interface to Miro internals

Ideas for extensions (these may not be possible, yet):

  1. site-specific support (e.g. YouTube urls, titles, and enclosureless-feeds)
  2. video search engines (e.g. Bing video search)
  3. extension for scheduling times when the downloader can be downloading -- helps in places where download bandwidth pricing changes during the day
  4. extension for keeping track of total amount downloaded and shuts off downloader when a threshold is reached (davidstoll suggestion)
  5. extensions for adding device syncing support for device XXX

FIXME - add additional requirements, use cases and user stories here


Research Links

Extensions in Python:

Extension index sites:


Design Thoughts

This is a huge project with a lot of pieces to it--we're not going to implement everything in one version.

This document will cover the infrastructure involved. It will not cover the Extensions API--that is covered in ExtensionsAPI.

Architecture

ExtensionManager

The ExtensionManager manages extensions in Miro. More specifically, it will:

  1. return a list of all extensions it knows about
  2. return whether a given extension is enabled/disabled
  3. enable an extension
  4. disable an extension
  5. load an extension
  6. unload an extension (as much as possible)
  7. handle load/unload/enable/disable errors gracefully
  8. report load/unload/enable/disable errors appropriately (log file? dialog?)

Miro core extensions will be located in tv/extensions/ in separate directories.

Currently, it looks like this:

- tv/
  |- extensions/
     |- README                    documentation for directory
     |- __init__.py               makes this directory a Python package
     |- testharness.py            testharness for extension tests
     | 
     |- watchhistory.miroext      sample extension
     |- watchhistory/
        |- __init__.py
        |- main.py

The user's list of enabled extensions is persisted to config.

The user's list of disabled extensions is also persisted to config.

If Miro loads and sees an extensions that's in neither the enabled or disabled list, then it checks the extension to see if it should be enabled by default and enables it accordingly.


miro.plat.resources.extension_roots

Each platform will have additional places for extensions which allow users to install their own extensions. miro.plat.resources.extension_roots function should return a list of platform-specific directories to search for extensions in. Example from GNU/Linux:

  def extension_roots():
      return [os.path.join(root(), 'extensions'), "%(supportdir)s/extensions"]

This allows for variable expansion for the following variables:


GNU/Linux


OSX

FIXME - is this the right place to put extensions on OSX?


Windows


Anatomy of an Extension

The structure for an extension needs to support the following requirements:

  1. Miro needs to be able to query extension metadata (version, title, description, ...) WITHOUT importing a Python file
  2. support multiple Python modules, packages, and resource files
  3. support enabled-by-default for core extensions only
  4. allow for extensions to be developed in-place (i.e. a directory tree of files)
  5. allow for an extension to be moved around as a single unit (e.g. a .tar.gz file, an .egg file, ...)

To support these requirements, we require at least a .miroext file and either a Python module or package.

The .miroext file is a config file with a [extension] section and a few pieces of metadata.

config variable datatype purpose
name string The name/title of the extension
version string The version number for this extension. core if the extension comes with Miro or some version string (e.g. 1.0, 2.5, 1.1.1, ...)
description string The description of this extension. This can be multi-lined. See the examples below.
enabled_by_default True/False Whether or not this extension should be enabled by default. This only affects the first time Miro is started after the extension has been discovered/installed.
module string The Python module that holds the load and unload functions.

Here's an example:

[extension]
name = Watch History
version = core
description = Captures what you watched in a CSV file.
enabled_by_default = True
module = watchhistory

Example of an example with a multi-lined description:

[extension]
name = Watch History
version = core
description = This is a test extension that shows basic
    structure for what extensions should look like.
    .
    It's otherwise not very interesting.
    .
    Every time you play an item, Watch History records the
    name of the item you played and the time you played it
    in a CSV file in your support directory.
enabled_by_default = True
module = watchhistory

This works with this directory tree:

  |- watchhistory.miroext
  |- watchhistory/
     |- __init__.py         has the load/unload functions
     |- main.py             has the rest of the code referred to in __init__.py


__init__.py:

from watchhistory import main

WATCHER = None

def load(context):
    """Loads the watchhistory module.
    """
    global WATCHER
    WATCHER = main.WatchHistory()

def unload():
    pass

main.py:

import logging
import os
import time
import csv

from miro import app
from miro import api

my_logger = logging.getLogger('watchhistory')

class WatchHistory():
    def __init__(self):
        self.csv_writer = None
        self.item_info = None
        api.signals.system.connect(
            'startup-success', self.handle_startup_success)

    def handle_startup_success(self, obj):
        my_logger.info("startup")
        # open log file
        logfile = os.path.join(api.get_support_directory(), "watched.csv")
        fp = open(logfile, "ab")
        self.csv_writer = csv.writer(fp)

        # connect to will-play
        app.playback_manager.connect('selecting-file', self.handle_select_file)
        app.playback_manager.connect('will-play', self.handle_will_play)

    def handle_select_file(self, obj, item_info):
        self.item_info = item_info

    def handle_will_play(self, obj, duration):
        if self.csv_writer and self.item_info:
            row = [
                time.ctime(),
                self.item_info.name,

                self.item_info.duration]
            self.csv_writer.writerow(row)
            # we wipe out self.item_info because will-play gets called
            # whenver someone starts watching an item (which we want
            # to log), but also whenever someone pauses and plays an
            # item (which we don't want to log)
            self.item_info = None

Note:

The API used above is in flux!


See tv/extensions/ in the repository at https://git.participatoryculture.org/miro/ for examples.

Preferences Panel

The preference panel should allow the following:

  1. enable/disable existing extensions
  2. configure extensions
  3. install new extensions
  4. upgrade existing extensions

FIXME - The ui here needs work. For now it looks a lot like the watched folders panel and only allows the user to enable/disable existing extensions.


Future directions

Installing new extensions

The user should be able to:

  1. go to the extensions web-site, browse, comment on, rate and install new extensions
  2. remove user-installed extensions

Status: For now, there is no web-site. Users will have to manually install and remove extensions by untarring relevant files in their platform-appropriate user directories. There's a lot of work that needs to be done here.


Upgrading extensions

Miro should be able to:

  1. query a web-site for new versions of installed extensions
  2. upgrade the extensions in-place
  3. manage multiple versions of extensions

Status: Need other infrastructure before we're going to attempt to implement this.


Order of extension loading

If it becomes necessary to load extensions in a specific order, then we should probably implement dependency-based load order figuring. Extensions would need to specify what dependencies they have and Miro would graph it all and figure out what the load order would be.

Status: Kick this can down the road until there's a compelling reason for it.


API

Extensions need to be able to access Miro internals. The Extensions API allows for this by creating a shim making it easier for extension writers to write extensions and easier for Miro developers to change internals without affecting extensions.

The Extensions API is covered on ExtensionsAPI.


Frequently Asked Questions

Can I do xyz with extensions?

Ask on #miro-hackers or on the develop mailing list. See MiroStart for details.


Do extensions run in a sandbox?

No, they don't. Extensions run in the same process space with the same permissions as the Miro process.


Etc

FIXME - Design details here. Include mockups, relevant bugs, ...

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox