Skip to main content

External REST APIs

The RESTApi class allows you to consume external REST APIs and convert responses to Arquero tables. Useful for integrating third-party data sources into Farseer workflows.

Creating an Instance​

Header Authentication​

import { RESTApi } from 'farseer-client';

const api = await RESTApi.createInstance(
    'https://api.example.com',
    {
        headerKey: 'Authorization',
        headerValue: 'Bearer my-token'
    }
);

Query Parameter Authentication​

const api = await RESTApi.createInstance(
    'https://api.example.com',
    {
        paramKey: 'api_key',
        paramValue: 'my-api-key'
    }
);

Login Authentication​

const api = await RESTApi.createInstance(
    'https://api.example.com',
    {
        request: {
            url: '/auth/login',
            method: 'POST',
            data: { username: 'user', password: 'pass' }
        },
        headerParamToExtract: {
            keyToRead: 'token',
            keyToWrite: 'Authorization',
            valueInitial: 'Bearer ',
            target: 'headers'
        }
    }
);

Hosts​

A RESTApi sends requests only to the host of its base URL, and only over https when the base URL uses https, so its credentials can't reach another host through an absolute URL or a redirect. A request or redirect to another host, or from https to plain http, is refused with an error. To call another host, create a separate RESTApi for it. The login request is the exception: it may go to another host, such as an OAuth2 token endpoint, and its redirects stay on that host.

Fetching and Converting Data​

Basic Request​

const data = await api.requestAndConvert({
    mainRequest: {
        url: '/api/products',
        method: 'GET'
    },
    responseTransform: ['data', 'items']  // Navigate: response.data.items
});

data.print();

With Transform Function​

For complex response structures, use a transform function:

const data = await api.requestAndConvert({
    mainRequest: {
        url: '/api/sales',
        method: 'POST',
        data: { year: 2024, region: 'North' }
    },
    responseTransform: (response) => {
        return response?.data?.records?.map(r => ({
            product: r.product_name,
            amount: r.total_amount,
            date: r.sale_date
        }));
    }
});

With Pagination​

Automatically handle paginated APIs:

const data = await api.requestAndConvert({
    mainRequest: {
        url: '/api/transactions',
        method: 'GET'
    },
    responseTransform: ['data'],
    paginationSettings: {
        paramHandlingType: 'query',
        limitParamName: 'limit',
        offsetParamName: 'offset',
        limitParamValue: 100
    }
});

With Pre-Request​

Fetch a token or CSRF before the main request:

const data = await api.requestAndConvert({
    mainRequest: {
        url: '/api/data',
        method: 'GET'
    },
    responseTransform: ['results'],
    preRequest: {
        requestConfig: {
            url: '/api/csrf-token',
            method: 'GET'
        },
        paramToExtract: {
            keyToRead: 'csrfToken',
            keyToWrite: 'X-CSRF-Token',
            target: 'headers'
        }
    }
});

Full Workflow: External API to Farseer Import​

import { RESTApi, FarseerClient } from 'farseer-client';

async function syncFromExternalAPI() {
    // 1. Connect to external API
    const api = await RESTApi.createInstance(
        'https://erp.company.com/api',
        { headerKey: 'Authorization', headerValue: 'Bearer token' }
    );

    // 2. Fetch data
    const salesData = await api.requestAndConvert({
        mainRequest: { url: '/sales', method: 'GET' },
        responseTransform: ['data'],
        paginationSettings: {
            paramHandlingType: 'query',
            limitParamName: 'limit',
            offsetParamName: 'offset',
            limitParamValue: 500
        }
    });

    // 3. Import into Farseer
    const client = new FarseerClient();
    const importJob = await client.createImportJob({
        title: 'ERP Sales Sync',
        columns: [
            { type: 'DIMENSION_TABLE', dimensionTableName: 'Products' },
            { type: 'DIMENSION_TABLE', dimensionTableName: 'Years' },
            { type: 'DIMENSION_TABLE', dimensionTableName: 'Months' },
            { type: 'VARIABLE', variableName: 'Revenue' },
        ],
        labels: ['auto', 'erp-sync', 'sales']
    });

    const rows = salesData.objects().map((r: any) => [
        r.product, r.year, r.month, r.amount
    ]);

    await importJob.addRows(rows);
    await importJob.flushRows();
    await importJob.undoPrevious();
    await importJob.commit();
}