bernhard/freifunk-app
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

initial commit taken from gitlab.lrz.de

Browse Source
This commit is contained in:
privatereese
2018-08-24 18:09:42 +02:00
parent ae54ed4c48
commit fc05486403
28494 changed files with 2159823 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

234
node_modules/jest-worker/README.md generated vendored Normal file
View File

@@ -0,0 +1,234 @@
# jest-worker
Module for executing heavy tasks under forked processes in parallel, by
providing a `Promise` based interface, minimum overhead, and bound workers.
The module works by providing an absolute path of the module to be loaded in all
forked processes. Files relative to a node module are also accepted. All methods
are exposed on the parent process as promises, so they can be `await`'ed. Child
(worker) methods can either be synchronous or asynchronous.
The module also implements support for bound workers. Binding a worker means
that, based on certain parameters, the same task will always be executed by the
same worker. The way bound workers work is by using the returned string of the
`computeWorkerKey` method. If the string was used before for a task, the call
will be queued to the related worker that processed the task earlier; if not, it
will be executed by the first available worker, then sticked to the worker that
executed it; so the next time it will be processed by the same worker. If you
have no preference on the worker executing the task, but you have defined a
`computeWorkerKey` method because you want _some_ of the tasks to be sticked,
you can return `null` from it.
The list of exposed methods can be explicitly provided via the `exposedMethods`
option. If it is not provided, it will be obtained by requiring the child module
into the main process, and analyzed via reflection. Check the "minimal example"
section for a valid one.
## Install
```sh
$ yarn add jest-worker
```
## Example
This example covers the minimal usage:
### File `parent.js`
```javascript
import Worker from 'jest-worker';
async function main() {
const worker = new Worker(require.resolve('./worker'));
const result = await worker.hello('Alice'); // "Hello, Alice"
}
main();
```
### File `worker.js`
```javascript
export function hello(param) {
return 'Hello, ' + param;
}
```
## API
The only exposed method is a constructor (`Worker`) that is initialized by
passing the worker path, plus an options object.
### `workerPath: string` (required)
Node module name or absolute path of the file to be loaded in the child
processes. Use `require.resolve` to transform a relative path into an absolute
one.
### `options: Object` (optional)
#### `exposedMethods: $ReadOnlyArray<string>` (optional)
List of method names that can be called on the child processes from the parent
process. You cannot expose any method named like a public `Worker` method, or
starting with `_`. If you use method auto-discovery, then these methods will not
be exposed, even if they exist.
#### `numWorkers: number` (optional)
Amount of workers to spawn. Defaults to the number of CPUs minus 1.
#### `maxRetries: number` (optional)
Maximum amount of times that a dead child can be re-spawned, per call. Defaults
to `3`, pass `Infinity` to allow endless retries.
#### `forkOptions: Object` (optional)
Allow customizing all options passed to `childProcess.fork`. By default, some
values are set (`cwd`, `env` and `execArgv`), but you can override them and
customize the rest. For a list of valid values, check
[the Node documentation](https://nodejs.org/api/child_process.html#child_process_child_process_fork_modulepath_args_options).
#### `computeWorkerKey: (method: string, ...args: Array<any>) => ?string` (optional)
Every time a method exposed via the API is called, `computeWorkerKey` is also
called in order to bound the call to a worker. This is useful for workers that
are able to cache the result or part of it. You bound calls to a worker by
making `computeWorkerKey` return the same identifier for all different calls. If
you do not want to bind the call to any worker, return `null`.
The callback you provide is called with the method name, plus all the rest of
the arguments of the call. Thus, you have full control to decide what to return.
Check a practical example on bound workers under the "bound worker usage"
section.
By default, no process is bound to any worker.
## Worker
The returned `Worker` instance has all the exposed methods, plus some additional
ones to interact with the workers itself:
### `getStdout(): Readable`
Returns a `ReadableStream` where the standard output of all workers is piped.
Note that the `silent` option of the child workers must be set to `true` to make
it work. This is the default set by `jest-worker`, but keep it in mind when
overriding options through `forkOptions`.
### `getStderr(): Readable`
Returns a `ReadableStream` where the standard error of all workers is piped.
Note that the `silent` option of the child workers must be set to `true` to make
it work. This is the default set by `jest-worker`, but keep it in mind when
overriding options through `forkOptions`.
### `end()`
Finishes the workers by killing all workers. No further calls can be done to the
`Worker` instance.
**Note:** Each worker has a unique id (index that starts with `1`) which is
available on `process.env.JEST_WORKER_ID`
# More examples
## Standard usage
This example covers the standard usage:
### File `parent.js`
```javascript
import Worker from 'jest-worker';
async function main() {
const myWorker = new Worker(require.resolve('./worker'), {
exposedMethods: ['foo', 'bar', 'getWorkerId'],
numWorkers: 4,
});
console.log(await myWorker.foo('Alice')); // "Hello from foo: Alice"
console.log(await myWorker.bar('Bob')); // "Hello from bar: Bob"
console.log(await myWorker.getWorkerId()); // "3" -> this message has sent from the 3rd worker
myWorker.end();
}
main();
```
### File `worker.js`
```javascript
export function foo(param) {
return 'Hello from foo: ' + param;
}
export function bar(param) {
return 'Hello from bar: ' + param;
}
export function getWorkerId() {
return process.env.JEST_WORKER_ID;
}
```
## Bound worker usage:
This example covers the usage with a `computeWorkerKey` method:
### File `parent.js`
```javascript
import Worker from 'jest-worker';
async function main() {
const myWorker = new Worker(require.resolve('./worker'), {
computeWorkerKey: (method, filename) => filename,
});
// Transform the given file, within the first available worker.
console.log(await myWorker.transform('/tmp/foo.js'));
// Wait a bit.
await sleep(10000);
// Transform the same file again. Will immediately return because the
// transformed file is cached in the worker, and `computeWorkerKey` ensures
// the same worker that processed the file the first time will process it now.
console.log(await myWorker.transform('/tmp/foo.js'));
myWorker.end();
}
main();
```
### File `worker.js`
```javascript
import babel from 'babel-core';
const cache = Object.create(null);
export function transform(filename) {
if (cache[filename]) {
return cache[filename];
}
// jest-worker can handle both immediate results and thenables. If a
// thenable is returned, it will be await'ed until it resolves.
return new Promise((resolve, reject) => {
babel.transformFile(filename, (err, result) => {
if (err) {
reject(err);
} else {
resolve((cache[filename] = result));
}
});
});
}
```

104
node_modules/jest-worker/build/child.js generated vendored Normal file
View File

@@ -0,0 +1,104 @@
/**
* Copyright (c) 2017-present, Facebook, Inc. All rights reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*
*
*/
'use strict';var _types;
function _load_types() {return _types = require('./types');}
let file = null;
/**
* This file is a small bootstrapper for workers. It sets up the communication
* between the worker and the parent process, interpreting parent messages and
* sending results back.
*
* The file loaded will be lazily initialized the first time any of the workers
* is called. This is done for optimal performance: if the farm is initialized,
* but no call is made to it, child Node processes will be consuming the least
* possible amount of memory.
*
* If an invalid message is detected, the child will exit (by throwing) with a
* non-zero exit code.
*/
process.on('message', (request /* Should be ChildMessage */) => {
switch (request[0]) {
case (_types || _load_types()).CHILD_MESSAGE_INITIALIZE:
file = request[2];
break;
case (_types || _load_types()).CHILD_MESSAGE_CALL:
execMethod(request[2], request[3]);
break;
case (_types || _load_types()).CHILD_MESSAGE_END:
process.exit(0);
break;
default:
throw new TypeError(
'Unexpected request from parent process: ' + request[0]);}
});
function reportSuccess(result) {
if (!process || !process.send) {
throw new Error('Child can only be used on a forked process');
}
process.send([(_types || _load_types()).PARENT_MESSAGE_OK, result]);
}
function reportError(error) {
if (!process || !process.send) {
throw new Error('Child can only be used on a forked process');
}
if (error == null) {
error = new Error('"null" or "undefined" thrown');
}
process.send([(_types || _load_types()).PARENT_MESSAGE_ERROR,
error.constructor && error.constructor.name,
error.message,
error.stack,
// $FlowFixMe: this is safe to just inherit from Object.
typeof error === 'object' ? Object.assign({}, error) : error]);
}
function execMethod(method, args) {
// $FlowFixMe: This has to be a dynamic require.
const main = require(file);
let result;
try {
if (method === 'default') {
result = (main.__esModule ? main['default'] : main).apply(global, args);
} else {
result = main[method].apply(main, args);
}
} catch (err) {
reportError(err);
return;
}
if (result && typeof result.then === 'function') {
result.then(reportSuccess, reportError);
} else {
reportSuccess(result);
}
}

204
node_modules/jest-worker/build/index.js generated vendored Normal file
View File

@@ -0,0 +1,204 @@
/**
* Copyright (c) 2017-present, Facebook, Inc. All rights reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*
*
*/
'use strict';Object.defineProperty(exports, "__esModule", { value: true });var _mergeStream;
function _load_mergeStream() {return _mergeStream = _interopRequireDefault(require('merge-stream'));}var _os;
function _load_os() {return _os = _interopRequireDefault(require('os'));}var _path;
function _load_path() {return _path = _interopRequireDefault(require('path'));}var _types;
function _load_types() {return _types = require('./types');}var _worker;
function _load_worker() {return _worker = _interopRequireDefault(require('./worker'));}function _interopRequireDefault(obj) {return obj && obj.__esModule ? obj : { default: obj };}
/* istanbul ignore next */
const emptyMethod = () => {};
/**
* The Jest farm (publicly called "Worker") is a class that allows you to queue
* methods across multiple child processes, in order to parallelize work. This
* is done by providing an absolute path to a module that will be loaded on each
* of the child processes, and bridged to the main process.
*
* Bridged methods are specified by using the "exposedMethods" property of the
* options "object". This is an array of strings, where each of them corresponds
* to the exported name in the loaded module.
*
* You can also control the amount of workers by using the "numWorkers" property
* of the "options" object, and the settings passed to fork the process through
* the "forkOptions" property. The amount of workers defaults to the amount of
* CPUS minus one.
*
* Queueing calls can be done in two ways:
* - Standard method: calls will be redirected to the first available worker,
* so they will get executed as soon as they can.
*
* - Sticky method: if a "computeWorkerKey" method is provided within the
* config, the resulting string of this method will be used as a key.
* Everytime this key is returned, it is guaranteed that your job will be
* processed by the same worker. This is specially useful if your workers are
* caching results.
*/exports.default =
class {
constructor(workerPath) {let options = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : {};
const numWorkers = options.numWorkers || (_os || _load_os()).default.cpus().length - 1;
const workers = new Array(numWorkers);
const stdout = (0, (_mergeStream || _load_mergeStream()).default)();
const stderr = (0, (_mergeStream || _load_mergeStream()).default)();
if (!(_path || _load_path()).default.isAbsolute(workerPath)) {
workerPath = require.resolve(workerPath);
}
const sharedWorkerOptions = {
forkOptions: options.forkOptions || {},
maxRetries: options.maxRetries || 3,
workerPath };
for (let i = 0; i < numWorkers; i++) {
const workerOptions = Object.assign({}, sharedWorkerOptions, {
workerId: i + 1 });
const worker = new (_worker || _load_worker()).default(workerOptions);
const workerStdout = worker.getStdout();
const workerStderr = worker.getStderr();
if (workerStdout) {
stdout.add(workerStdout);
}
if (workerStderr) {
stderr.add(workerStderr);
}
workers[i] = worker;
}
let exposedMethods = options.exposedMethods;
// If no methods list is given, try getting it by auto-requiring the module.
if (!exposedMethods) {
// $FlowFixMe: This has to be a dynamic require.
const child = require(workerPath);
exposedMethods = Object.keys(child).filter(
name => typeof child[name] === 'function');
if (typeof child === 'function') {
exposedMethods.push('default');
}
}
exposedMethods.forEach(name => {
if (name.startsWith('_')) {
return;
}
if (this.constructor.prototype.hasOwnProperty(name)) {
throw new TypeError('Cannot define a method called ' + name);
}
// $FlowFixMe: dynamic extension of the class instance is expected.
this[name] = this._makeCall.bind(this, name);
});
this._stdout = stdout;
this._stderr = stderr;
this._ending = false;
this._cacheKeys = Object.create(null);
this._options = options;
this._workers = workers;
this._offset = 0;
}
getStdout() {
return this._stdout;
}
getStderr() {
return this._stderr;
}
end() {
if (this._ending) {
throw new Error('Farm is ended, no more calls can be done to it');
}
const workers = this._workers;
// We do not cache the request object here. If so, it would only be only
// processed by one of the workers, and we want them all to close.
for (let i = 0; i < workers.length; i++) {
workers[i].send([(_types || _load_types()).CHILD_MESSAGE_END, false], emptyMethod);
}
this._ending = true;
}
// eslint-disable-next-line no-unclear-flowtypes
_makeCall(method) {for (var _len = arguments.length, args = Array(_len > 1 ? _len - 1 : 0), _key = 1; _key < _len; _key++) {args[_key - 1] = arguments[_key];}
if (this._ending) {
throw new Error('Farm is ended, no more calls can be done to it');
}
return new Promise((resolve, reject) => {const
computeWorkerKey = this._options.computeWorkerKey;
const workers = this._workers;
const length = workers.length;
const cacheKeys = this._cacheKeys;
const request = [(_types || _load_types()).CHILD_MESSAGE_CALL, false, method, args];
let worker = null;
let hash = null;
if (computeWorkerKey) {
hash = computeWorkerKey.apply(this, [method].concat(args));
worker = hash == null ? null : cacheKeys[hash];
}
// Do not use a fat arrow since we need the "this" value, which points to
// the worker that executed the call.
function callback(error, result) {
if (hash != null) {
cacheKeys[hash] = this;
}
if (error) {
reject(error);
} else {
resolve(result);
}
}
// If a worker is pre-selected, use it...
if (worker) {
worker.send(request, callback);
return;
}
// ... otherwise use all workers, so the first one available will pick it.
for (let i = 0; i < length; i++) {
workers[(i + this._offset) % length].send(request, callback);
}
this._offset++;
});
}};

95
node_modules/jest-worker/build/types.js generated vendored Normal file
View File

@@ -0,0 +1,95 @@
/**
* Copyright (c) 2017-present, Facebook, Inc. All rights reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*
*
*/
'use strict';
// Because of the dynamic nature of a worker communication process, all messages
// coming from any of the other processes cannot be typed. Thus, many types
// include "any" as a flow type, which is (unfortunately) correct here.
/* eslint-disable no-unclear-flowtypes */Object.defineProperty(exports, "__esModule", { value: true });
const CHILD_MESSAGE_INITIALIZE = exports.CHILD_MESSAGE_INITIALIZE = 0;
const CHILD_MESSAGE_CALL = exports.CHILD_MESSAGE_CALL = 1;
const CHILD_MESSAGE_END = exports.CHILD_MESSAGE_END = 2;
const PARENT_MESSAGE_OK = exports.PARENT_MESSAGE_OK = 0;
const PARENT_MESSAGE_ERROR = exports.PARENT_MESSAGE_ERROR = 1;
// Option objects.
// Messages passed from the parent to the children.
// Messages passed from the children to the parent.
// Queue types.

193
node_modules/jest-worker/build/worker.js generated vendored Normal file
View File

@@ -0,0 +1,193 @@
/**
* Copyright (c) 2017-present, Facebook, Inc. All rights reserved.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*
*
*/
'use strict';Object.defineProperty(exports, "__esModule", { value: true });var _child_process;
function _load_child_process() {return _child_process = _interopRequireDefault(require('child_process'));}var _types;
function _load_types() {return _types = require('./types');}function _interopRequireDefault(obj) {return obj && obj.__esModule ? obj : { default: obj };}
/**
* This class wraps the child process and provides a nice interface to
* communicate with. It takes care of:
*
* - Re-spawning the process if it dies.
* - Queues calls while the worker is busy.
* - Re-sends the requests if the worker blew up.
*
* The reason for queueing them here (since childProcess.send also has an
* internal queue) is because the worker could be doing asynchronous work, and
* this would lead to the child process to read its receiving buffer and start a
* second call. By queueing calls here, we don't send the next call to the
* children until we receive the result of the previous one.
*
* As soon as a request starts to be processed by a worker, its "processed"
* field is changed to "true", so that other workers which might encounter the
* same call skip it.
*/exports.default =
class {
constructor(options) {
this._options = options;
this._queue = [];
this._initialize();
}
getStdout() {
return this._child.stdout;
}
getStderr() {
return this._child.stderr;
}
send(request, callback) {
this._queue.push({ callback, request });
this._process();
}
_initialize() {
const child = (_child_process || _load_child_process()).default.fork(
require.resolve('./child'),
// $FlowFixMe: Flow does not work well with Object.assign.
Object.assign(
{
cwd: process.cwd(),
env: Object.assign({}, process.env, {
JEST_WORKER_ID: this._options.workerId }),
// suppress --debug / --inspect flags while preserving others (like --harmony)
execArgv: process.execArgv.filter(v => !/^--(debug|inspect)/.test(v)),
silent: true },
this._options.forkOptions));
child.on('message', this._receive.bind(this));
child.on('exit', this._exit.bind(this));
// $FlowFixMe: wrong "ChildProcess.send" signature.
child.send([(_types || _load_types()).CHILD_MESSAGE_INITIALIZE, false, this._options.workerPath]);
this._retries++;
this._child = child;
this._busy = false;
// If we exceeded the amount of retries, we will emulate an error reply
// coming from the child. This avoids code duplication related with cleaning
// the queue, and scheduling the next call.
if (this._retries > this._options.maxRetries) {
const error = new Error('Call retries were exceeded');
this._receive([(_types || _load_types()).PARENT_MESSAGE_ERROR,
error.name,
error.message,
error.stack,
{ type: 'WorkerError' }]);
}
}
_process() {
if (this._busy) {
return;
}
const queue = this._queue;
let skip = 0;
// Calls in the queue might have already been processed by another worker,
// so we have to skip them.
while (queue.length > skip && queue[skip].request[1]) {
skip++;
}
// Remove all pieces at once.
queue.splice(0, skip);
if (queue.length) {
const call = queue[0];
// Flag the call as processed, so that other workers know that they don't
// have to process it as well.
call.request[1] = true;
this._retries = 0;
this._busy = true;
// $FlowFixMe: wrong "ChildProcess.send" signature.
this._child.send(call.request);
}
}
_receive(response /* Should be ParentMessage */) {
const callback = this._queue[0].callback;
this._busy = false;
this._process();
switch (response[0]) {
case (_types || _load_types()).PARENT_MESSAGE_OK:
callback.call(this, null, response[1]);
break;
case (_types || _load_types()).PARENT_MESSAGE_ERROR:
let error = response[4];
if (error != null && typeof error === 'object') {
const extra = error;
const NativeCtor = global[response[1]];
const Ctor = typeof NativeCtor === 'function' ? NativeCtor : Error;
error = new Ctor(response[2]);
// $FlowFixMe: adding custom properties to errors.
error.type = response[1];
error.stack = response[3];
for (const key in extra) {
// $FlowFixMe: adding custom properties to errors.
error[key] = extra[key];
}
}
callback.call(this, error, null);
break;
default:
throw new TypeError('Unexpected response from worker: ' + response[0]);}
}
_exit(exitCode) {
if (exitCode !== 0) {
this._initialize();
}
}};

80
node_modules/jest-worker/package.json generated vendored Normal file
View File

@@ -0,0 +1,80 @@
{
"_args": [
[
"jest-worker@22.2.2",
"/home/bernhard/freifunk-app/node_modules/metro"
]
],
"_from": "jest-worker@22.2.2",
"_id": "jest-worker@22.2.2",
"_inCache": true,
"_installable": true,
"_location": "/jest-worker",
"_nodeVersion": "8.9.1",
"_npmOperationalInternal": {
"host": "s3://npm-registry-packages",
"tmp": "tmp/jest-worker_22.2.2_1518193692981_0.36704386466582917"
},
"_npmUser": {
"email": "mjesun@hotmail.com",
"name": "mjesun"
},
"_npmVersion": "5.5.1",
"_phantomChildren": {},
"_requested": {
"name": "jest-worker",
"raw": "jest-worker@22.2.2",
"rawSpec": "22.2.2",
"scope": null,
"spec": "22.2.2",
"type": "version"
},
"_requiredBy": [
"/jest-haste-map",
"/jest-runner",
"/jest/jest-cli",
"/metro"
],
"_resolved": "https://registry.npmjs.org/jest-worker/-/jest-worker-22.2.2.tgz",
"_shasum": "c1f5dc39976884b81f68ec50cb8532b2cbab3390",
"_shrinkwrap": null,
"_spec": "jest-worker@22.2.2",
"_where": "/home/bernhard/freifunk-app/node_modules/metro",
"bugs": {
"url": "https://github.com/facebook/jest/issues"
},
"dependencies": {
"merge-stream": "^1.0.1"
},
"description": "Module for executing heavy tasks under forked processes in parallel, by providing a `Promise` based interface, minimum overhead, and bound workers.",
"devDependencies": {},
"directories": {},
"dist": {
"fileCount": 6,
"integrity": "sha512-ZylDXjrFNt/OP6cUxwJFWwDgazP7hRjtCQbocFHyiwov+04Wm1x5PYzMGNJT53s4nwr0oo9ocYTImS09xOlUnw==",
"shasum": "c1f5dc39976884b81f68ec50cb8532b2cbab3390",
"tarball": "https://registry.npmjs.org/jest-worker/-/jest-worker-22.2.2.tgz",
"unpackedSize": 26203
},
"homepage": "https://github.com/facebook/jest#readme",
"license": "MIT",
"main": "build/index.js",
"maintainers": [
{
"name": "cpojer",
"email": "christoph.pojer@gmail.com"
},
{
"name": "mjesun",
"email": "mjesun@hotmail.com"
}
],
"name": "jest-worker",
"optionalDependencies": {},
"readme": "ERROR: No README data found!",
"repository": {
"type": "git",
"url": "git+https://github.com/facebook/jest.git"
},
"version": "22.2.2"
}