Revision as of 19:12, 13 August 2013

Description

The GPII/Kettle architecture is designed to be flexible and highly configurable. It consists of many components that can be configured without modifying their implementation, or swapped out entirely for other implementations. This configuration is done by creating new or editing existing JSON configuration files that describe various components for specific GPII/Kettle deployment types.

With the help of these configuration files, developers can configure which components are deployed and with what settings. Examples of such configurations include:

• Configuring URL locations for GPII resources
• Configuring Match Maker strategies
• Registering events and event listeners within various workflows

This page contains all the necessary information on how to configure various components (Flow Manager, Match Maker, Device Reporter, etc) within the GPII framework.

Getting Started

In order to proceed with the configuration and deployment tasks, please make sure that your development environment is set up: Setting Up Your Development Environment.

The source code for the core framework and all platform independent components can be found here: GPII Universal, Kettle

All platform specific components can be found in the following repositories:

• GPII Android
• GPII Linux
• GPII Windows

Kettle Server

The kettle.server component is one of the building blocks of the Kettle framework you are deploying. It represents a single instance of a Node.js http server.

It serves as a container for one or more kettle.apps (see below). Developers have the option of deploying a number of Kettle servers that themselves contain one or more Kettle apps.

The source code for the kettle.server can be found here: kettle.server

Kettle App

The kettle.app component is a component that represents a single purpose app deployed within a particular kettle.server. The majority of the existing components within the Kettle architecture that support the HTTP API (such as the Flow Manager, Device Reporter, Match Maker, Preferences Server, Solutions Registry, etc) are, in fact, kettle.apps themselves.

The source code for the kettle.app can be found here: kettle.app

To configure a specific Kettle deployment, you will be setting up the appropriate Kettle servers and Kettle apps required for the deployment.

Creating a Configuration File

There are several conventions in regards to where your configuration file should go:

• If you are creating configuration for multiple apps, consider putting it in universal/gpii/configs if all your Kettle apps are defined in a GPII Universal repository. If one or more apps are defined in a platform specific repository, your configuration files should go into a top level configs directory (Note: you might have to create it) within the appropriate platform specific repository.
• If you are creating configuration for a single kettle.app, you need to save it in the configs directory that corresponds to the location of that Kettle app. Currently, all Kettle apps are located in universal/gpii/node_modules (e.g. Preferences Server, Solutions Registry, etc). Thus, the config file should be located at this path: universal/gpii/node_modules/{some kettle app}/configs.

Basic configuration

A single kettle.server and no kettle.apps.

This configuration will deploy a single empty server container:

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {                // List of components to be deployed.
            "kettle server component": { // The name of the server component.
                "type": "kettle.server"  // The type of the server component.
            }
        }
    }
}

Multiple servers

Two kettle.servers with no kettle.apps.

This configuration will deploy two empty server containers.

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server one": {
                "type": "kettle.server"
            },
            "server two": {
                "type": "kettle.server"
            }
        }
    }
}

One server with one app

A single kettle.server that contains a kettle.app.

This configuration will deploy a single server with an empty app that does nothing.

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "components": {
                    "kettle app component": {
                        "type": "kettle.app" // For example: "gpii.preferencesServer"
                    }
                }
            }
        }
    }
}

One server with several apps

A single kettle.server that contains two kettle.apps.

This configuration will deploy a single server with two empty apps that do nothing.

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "components": {
                    "kettle app component": {
                        "type": "kettle.app"
                    },
                    "preferencesServer": {
                        "type": "gpii.preferencesServer"
                    }
                }
            }
        }
    }
}

Options

Each GPII component is capable of interpreting optional parameters passed to it (these options are meant to override the default setting found in the actual component definition).

• This configuration will deploy a server on port 80 with logging enabled

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "kettle server component": {
                "type": "kettle.server",
                "options": {
                    // Kettle Server options
                    "logging": true,
                    "port": 80
                }
            }
        }
    }
}

• This configuration will deploy a server with an app that has "some option" setting set to "some option value"

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "components": {
                    "kettle app component": {
                        "type": "kettle.app",
                        "options": {
                            "some option": "some option value" // Kettle App option
                        }
                    }
                }
            }
        }
    }
}

Includes

Configuration files are capable of including other existing configuration files to avoid duplication.

This configuration will deploy a server and also load and process the contents of all configuration files mentioned in 'includes' section.

{
    "typeName": "your configuration name",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server"
            }
        }
    },
    "includes": [
        "relative path to another config file"
    ]
}

Demands (Advanced)

There is another indirect way of configuring all components in the deployment that are not necessarily a kettle.server or a kettle.app. It is done via specific configuration blocks called "demands".

This configuration will deploy a server and also set a "some option" setting set to "some option value" for a "component name we want to configure" when it is created as part of "your configuration name" configuration.

{
    "typeName": "your configuration name", // (*)
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server"
            }
        }
    },
    "demands": [{
        "demandingName": "component name we want to configure",
        // Context in which the component is initialized, in this case
        // it is under the "your configuration name" (*) context.
        "contextNames": "your configuration name",
        "demandSpec": {
            // Component configuration goes here:
            "options": {
                "some option": "some option value"
            }
        }
    }]
}

For more details on how to write demands blocks, see Demands Specifications

Examples of Existing Configuration Files

Below are examples of some of the existing configuration files that are currently used to deploy single Kettle apps or all of GPII architecture.

Preferences Server (Production)

This configuration deploys a preferences server app in a single server container (with logging disabled). We also configure the location of the users store to be at "http://localhost:5984/user/%token". Type of the user store is set to Couch DB (*).

{
    "typeName": "preferencesServer.production",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "options": {
                    "logging": false,
                    "components": {
                        "preferencesServer": {
                            "type": "gpii.preferencesServer",
                            "options": {
                                "userSourceUrl": "http://localhost:5984/user/%token"
                            }
                        }
                    }
                }
            }
        }
    },
    "demands": [{
        "demandingName": "rawUserSource",
        "contextNames": "gpii.preferencesServer",
        "demandSpec": {
            "funcName": "kettle.dataSource.CouchDB" // (*)
        }
    }],
    "includes": [
        "./base.json"
    ]
}

Preferences Server (Development)

This configuration deploys a preferences server app in a single server container (with logging enabled). We also configure the location of the users store to be at "file://%root/../../../testData/preferences/%token.json". Type of the user store is set to File (e.g. file path) (*).

{
    "typeName": "preferencesServer.development",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "options": {
                    "logging": true,
                    "components": {
                        "preferencesServer": {
                            "type": "gpii.preferencesServer",
                            "options": {
                                "userSourceUrl": "file://%root/../../../testData/preferences/%token.json"
                            }
                        }
                    }
                }
            }
        }
    },
    "demands": [{
        "demandingName": "kettle.urlExpander",
        "contextNames": "gpii.preferencesServer",
        "demandSpec": {
            "funcName": "kettle.urlExpander.development"
        }
    }, {
        "demandingName": "rawUserSource",
        "contextNames": "gpii.preferencesServer",
        "demandSpec": {
            "funcName": "kettle.dataSource.URL" // (*)
        }
    }],
    "includes": [
        "./base.json"
    ]
}

Preferences Server (base)

This configuration defines request handlers specification common between all types of preferences servers.

{
    "demands": [{
        "demandingName": "kettle.requests.request.handler",
        "contextNames": "userGet",
        "demandSpec": {
            "options": {
                "invokers": {
                    "handle": {
                        "funcName": "kettle.requests.request.handler.userGet",
                        "args": ["{requestProxy}", "{request}.req.params.token", "{preferencesServer}.userSource"]
                    }
                }
            }
        }
    }, {
        "demandingName": "kettle.requests.request.handler",
        "contextNames": "userPost",
        "demandSpec": {
            "options": {
                "invokers": {
                    "handle": {
                        "funcName": "kettle.requests.request.handler.userPost",
                        "args": ["{requestProxy}", "{request}.req", "{preferencesServer}.userSource"]
                    }
                }
            }
        }
    }, {
        "demandingName": "userSource",
        "contextNames": "gpii.preferencesServer",
        "demandSpec": {
            "options": {
                "components": {
                    "rawSource": "{rawUserSource}"
                }
            }
        }
    }],
    "modules": [
        "preferencesServer"
    ]
}

Match Maker (Development)

This configuration deploys a match maker app in a single server container (with logging enabled). We also configure the locations of the solutions registry to be at "file://%root/../../../testData/solutions/%os.json" and the ontology server to be at "file://%root/../../../testData/ontologies/ontologies.json".

{
    "typeName": "matchMaker.development",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "options": {
                    "logging": true,
                    "components": {
                        "matchMaker": {
                            "type": "gpii.matchMaker",
                            "options": {
                                "solutionsReporterUrl": "file://%root/../../../testData/solutions/%os.json",
                                "ontologyServerUrl": "file://%root/../../../testData/ontologies/ontologies.json"
                            }
                        }
                    }
                }
            }
        }
    },
    "demands": [{
        "demandingName": "kettle.urlExpander",
        "contextNames": "gpii.matchMaker",
        "demandSpec": {
            "funcName": "kettle.urlExpander.development"
        }
    }, {
        "demandingName": "rawOntologyServer",
        "contextNames": "gpii.matchMaker",
        "demandSpec": {
            "funcName": "gpii.ontologyServer.development"
        }
    }],
    "includes": [
        "./base.json"
    ]
}

All of GPII (Development)

We load and process all config files mentioned in 'includes' list and thus deploy a device reporter, a flow manager, a match maker, an ontology server, a preferences server and a solutions registry.

{
    "typeName": "flowManager.preferencesServer.solutionsRegistry.deviceReporter.matchMaker.ontologyServer.development",
    "options": {
        "gradeNames": ["autoInit", "fluid.littleComponent"],
        "components": {
            "server": {
                "type": "kettle.server",
                "options": {
                    "logging": true,
                    "port": 8081
                }
            }
        }
    },
    "includes": [
        "../node_modules/deviceReporter/configs/development.json",
        "../node_modules/flowManager/configs/development.json",
        "../node_modules/matchMaker/configs/development.json",
        "../node_modules/ontologyServer/configs/development.json",
        "../node_modules/preferencesServer/configs/development.json",
        "../node_modules/solutionsRegistry/configs/development.json"
    ]
}

Deploying GPII Architecture Components

Once the configuration file is ready, it is trivial to deploy the configuration on your platform.

Simply execute the following command in your terminal:

[NODE_ENV={environment}] node {universal}/node_modules/kettle/lib/init.js path/to/a/folder/with/your/config

Options

• environment should correspond to the name of your config file. Default value is 'development'
• universal is the location of the universal repository

Examples

• node node_modules/kettle/lib/init.js gpii/node_modules/preferencesServer/configs/
• NODE_ENV=production node node_modules/kettle/lib/init.js gpii/node_modules/matchMaker/configs/
• NODE_ENV=ps.sr.production node node_modules/kettle/lib/init.js gpii/configs/