# APISIX (Griaule Bus) ## Introduction Apache APISIX provides advanced traffic management features such as load balancing, dynamic upstream, circuit breaking, authentication, observability, etc. ## Manual Installation ### Installation {% hint style="warning" %} This installation method is only necessary if you need to install APISIX manually and use the API commands via the command line. Otherwise, use the Docker installation detailed in the [APISIX Dashboard](#apisix-dashboard-recomendado). {% endhint %} #### Installing *etcd* APISIX uses *etcd* to save and synchronize configuration. Before installing APISIX, you need to install *etcd* on your machine. It will be installed automatically if you choose the Docker or Helm installation method during APISIX installation. If you choose a different method or need to install it manually, follow the steps shown below: ```tsx ``` ```sh ETCD_VERSION='3.5.4' wget https://github.com/etcd-io/etcd/releases/download/v${ETCD_VERSION}/etcd-v${ETCD_VERSION}-linux-amd64.tar.gz tar -xvf etcd-v${ETCD_VERSION}-linux-amd64.tar.gz && \ cd etcd-v${ETCD_VERSION}-linux-amd64 && \ sudo cp -a etcd etcdctl /usr/bin/ nohup etcd >/tmp/etcd.log 2>&1 & ``` #### Installation via RPM package If the *OpenResty* is not installed, you can run the command below to install the *OpenResty* and APISIX repositories: ```sh sudo yum install -y https://repos.apiseven.com/packages/centos/apache-apisix-repo-1.0-1.noarch.rpm ``` With *OpenResty* installed, the command below will install the APISIX repositories: ```sh sudo yum-config-manager --add-repo https://repos.apiseven.com/packages/centos/apache-apisix.repo ``` Then, to install *APISIX*, use: ```sh sudo yum install apisix ``` {% hint style="success" %} You can also install a specific version of the APISIX package by specifying it explicitly: ```sh sudo yum install apisix-2.13.1 ``` {% endhint %} #### Offline RPM package installation First, download the offline RPM package for an APISIX repository: ```sh sudo mkdir -p apisix sudo yum install -y https://repos.apiseven.com/packages/centos/apache-apisix-repo-1.0-1.noarch.rpm sudo yum clean all && yum makecache sudo yum install -y --downloadonly --downloaddir=./apisix apisix ``` Then, copy the APISIX repository to the desired host and run: ```sh sudo yum install ./apisix/*.rpm ``` ### Managing the APISIX server Once APISIX has been installed, you can initialize the configuration file and the *etcd* by running the command: ```sh apisix init ``` To start the APISIX server, use: ```sh apisix start ``` {% hint style="success" %} The command `apisix help` returns a list of available operations and can be useful on various occasions. {% endhint %} If you want to build APISIX from source, see [Building APISIX from source](https://github.com/apache/apisix/blob/master/docs/en/latest/building-apisix.md). ### Configuring APISIX APISIX can be configured in two ways: 1. Directly changing the configuration file `conf/config.yaml`. 2. Using the command `--config` or the flag `-c` to specify the path to your configuration file while APISIX is being started: ```sh apisix start -c ``` APISIX will use the configurations added in that configuration file and will fall back to the default configuration if something is not configured. For example, to configure the default listening port to be `8000` without changing other settings, your configuration file should be modified as follows: ```yml apisix: node_listen: 8000 ``` Now, if you decide you want to change the *etcd* address to `http://foo:2379`, you can add it to your configuration file. This will not change other settings. ```yml apisix: node_listen: 8000 etcd: host: "http://foo:2379" ``` {% hint style="warning" %} Configuration should only be changed by the methods mentioned above. The default APISIX configuration can be found in the file `conf/config-default.yaml` and should not be modified. The default configuration is tied to the source code. {% endhint %} {% hint style="warning" %} The file `conf/nginx.conf` is generated automatically and should not be modified. {% endhint %} #### Updating the Admin API key It is recommended to change the Admin API key to ensure the application's security. To change this setting, simply modify the configuration file as shown below: ```sh apisix: admin_key - name: "admin" key: newsupersecurekey role: admin ``` With this, to access the Admin API, use the newly configured key: ```sh curl http://127.0.0.1:9080/apisix/admin/routes?api_key=newsupersecurekey -i ``` ### Adding the APISIX systemd unit file If you installed APISIX via RPM, the APISIX unit file will already be configured and you can start APISIX by: ```sh systemctl start apisix systemctl stop apisix ``` If you installed APISIX by other methods, you can create `/usr/lib/systemd/system/apisix.service` and add the configuration template. The [Getting Started](https://github.com/apache/apisix) page of APISIX provides a guide with instructions for properly using the API. ### Installed packages ``` +----------------------------------------------------------------+ | Package Arch Version Repository Size | +================================================================+ | Installing: | | apisix x86_64 2.14.1-0.el7 release 2.2 M | +----------------------------------------------------------------+ | Installing for dependencies: | | apisix-base x86_64 1.21.4.1.0-0.el7 release 34 M | +----------------------------------------------------------------+ | Updating for dependencies: | | cyrus-sasl-lib x86_64 2.1.26-24.el7_9 updates 156 k | | openldap x86_64 2.4.44-25.el7_9 updates 356 k | +----------------------------------------------------------------+ +-------------------------------------------+ | Transaction Summary | +===========================================+ | Install 1 Package (+7 Dependent packages) | | Upgrade (2 Dependent packages) | +-------------------------------------------+ | Total download size: 40 M | +-------------------------------------------+ ``` ## APISIX Dashboard (Recommended) ### Docker **It is recommended to use Docker to run the Dashboard**, using the commands: ```sh docker pull apache/apisix-dashboard docker run -d --name dashboard \ -p 9000:9000 \ -v :/usr/local/apisix-dashboard/conf/conf.yaml \ apache/apisix-dashboard ``` {% hint style="info" %} Replace `` with the path to your configuration file. {% endhint %} ### RPM Only necessary in the case of **NOT** using Docker. {% hint style="info" %} Only CentOS 7 is supported currently. {% endhint %} #### Installation Install the RPM package with the following command: ```sh sudo yum install -y https://github.com/apache/apisix-dashboard/releases/download/v2.13/apisix-dashboard-2.13-0.el7.x86_64.rpm ``` #### Running Run the Dashboard in the shell: ```sh sudo manager-api -p /usr/local/apisix/dashboard/ ``` or as a service: ```sh systemctl start apisix-dashboard ``` If the default configuration was used, visit `http://127.0.0.1:9000` to use the Dashboard. {% hint style="info" %} The default username and password are `admin`. {% endhint %} ## Access By default, the APISIX API will run on port `9080` of the machine. For example, on the machine `172.16.0.66`: `http://172.16.0.66:9080/apisix/admin/routes?api_key=newsupersecurekey` In summary, the services run by APISIX can be accessed on the following default ports: `9080`, `9082`, `2379`, `9000`, `3000`, `9090` and `9081`. ![apisix standard ports](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-c5b45302917122329dabb3809c52b3e35cb84865%2FapisixStandardPorts.png?alt=media) The APISIX Dashboard, which allows configuring the bus through a graphical interface, runs on port `9000`, for example. ## Configuring an API Web application APIs implement a variety of endpoints responsible for processing and returning information related to the applications' databases, the GBDS and performed operations. Most endpoints are application-specific, but some are common among them and are implemented in what is called the Common Server. All API calls used by the application must be configured in APISIX. There are 2 alternatives to configure an API, described below. ### Configuring the API via the APISIX Dashboard (Recommended) ![apisix dashboard home screen](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-daad070977dd73c56ba554e9885df4b8ce01bcd5%2FapisixDashboardHomeScreen.png?alt=media) Configuring the API through the APISIX Dashboard is recommended because the Dashboard's graphical interface allows importing the API via OpenAPI files in the `.json` or `.yaml`. To enable importing the API through the Dashboard, you must obtain the OpenAPI file of the application being configured. #### Generating the OpenAPI file from the server The web application servers implement Swagger for documenting their endpoints. To generate the API documentation file, just access the server and make a `GET` to the route `/service/swagger.yaml`. On the `ETR`, for example, just access: `http://172.16.0.70:8089/gbs-etr-server/service/swagger.yaml` When accessing this endpoint in the browser, for example, the file will be downloaded `.yaml`. To be imported into APISIX, this file must be converted to the OpenAPI 3.0 format. The simplest way to do this is to access the [Swagger Editor](https://editor.swagger.io) and follow these steps: 1. Import the generated file on the site by clicking File > Import file. 2. Convert to OpenAPI 3.0 by clicking Edit > Convert to OpenAPI 3.0. 3. Save the file in OpenAPI format (as `.yaml` or `.json`) by clicking File > Save as YAML or Convert and save as JSON. {% hint style="info" %} In practically all applications there are endpoints that end up not being used/called by the frontend and *pollute* the file. It is recommended that these endpoints be removed from the file, along with their respective *schemas* and components. {% endhint %} #### Importing the API via the Dashboard With APISIX running, go to `http://localhost:9000` and you will be redirected to the Dashboard. If login is required, by default access is done using the username and password `admin/admin`. Before importing the API routes, it is recommended to create an Upstream - which is basically the Backend Server of the API being configured. This step is done in the Upstream tab of the Dashboard. For this, you need to define a name for the server, the IP and the port at which it is running. Below is an example of the Upstream configuration for the `ETR` that is running on the machine `172.16.0.66`: ![apisix upstream configuration example](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-1b4eb491b5e7aad451cff307ba213c8aa225726c%2FapisixConfigureUpstreamExample.png?alt=media) Once the Upstream is configured, you can import the API routes using the generated OpenAPI 3.0 file: 1. Go to the Route tab. 2. Click the Advanced? dropdown menu. 3. Click Import OpenAPI. 4. Select the file and click the Confirm button. If everything is correct, the API routes will be imported into the dashboard and will be listed as in this example: ![apisix dashboard route list](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-ac4c98b3475f66361b67a38d2cb54f001944ab86%2FapisixDashboardRouteList.png?alt=media) {% hint style="success" %} If there are problems importing the OpenAPI file, consult the [FAQ](#faq). {% endhint %} #### Configuring the imported routes Once the routes have been imported by the Dashboard, some adjustments are still required so that the endpoints can be accessed by the applications. {% stepper %} {% step %} #### Mapping the routes The first point to be adjusted is the `path` of the route, that is, the address that will be accessed so that APISIX correctly identifies the endpoint call. {% hint style="info" %} By default, when generating the Swagger documentation for an application, the mapped routes will be relative to the server default. {% endhint %} On the `ETR`, for example, the endpoints generated are `/etr/list` or `/commons/version`. However, to access that endpoint through the web application or directly via command line, it is necessary to access the path `/gbs-etr-server/service/`. Therefore, it is necessary to change the attribute `path` of the imported routes and add the default route of the API being configured, including, for example, the path `gbs-API-server/service` (web applications' default) before the mapped routes. ![apisix map imported routes](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-1856b8900fb40c264e665273a5aaa7ab4a065ae6%2FapisixMapImportedRoutes.png?alt=media) {% endstep %} {% step %} #### Adding the HTTP OPTIONS method When importing the routes, they will be mapped only with the default HTTP method they execute. However, most calls made through web applications, by React's default, also perform an OPTIONS method - used so a client can discover which request options are allowed for a given resource - and if this is not mapped in APISIX, a CORS error may occur when the application makes the call. Therefore, it is important that the OPTIONS method be added to the routes to avoid API communication problems with APISIX. {% hint style="info" %} This step is necessary only to access the bus through other applications (Frontend/React, for example) - not necessary for direct access - and aims to resolve CORS (Cross-Origin Resource Sharing) issues, but it is not the only possible alternative. {% endhint %} ![apisix add http options](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-d9bb273d33b58d8ea9956a2eea3162a9bcf1a03c%2FapisixAddHttpOptions.png?alt=media) {% endstep %} {% step %} #### Defining the Upstream (Backend API Server) In addition to configuring the route and the expected methods, it is necessary to define the address of the server to which requests will be redirected by APISIX. This is the step where the previously defined Upstream is used. In the second configuration tab of the route, you can define the Backend API Server and, once the Upstream has been previously configured, you can select it via the dropdown menu at the top of the page. Just select the correct option and the calls will already be correctly configured, as shown in the image below: ![apisix route configuration](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-21348a271cc8be301034e3c763fa52e4ec917409%2FapisixConfigureRoute.png?alt=media) {% endstep %} {% step %} #### Configuring Plugins Next, you can configure the APISIX plugins that will be used by the route. By default, every imported route already has the `request-validation` plugin enabled, which validates the parameters sent with the request before forwarding it to the server. **It is recommended to remove this plugin** for all routes, since validation is already performed by the servers. There are several types of plugins that can be configured for routes to control/manage Authentication, Security, Traffic Control, among other aspects of the route, as can be seen on the configuration screen: ![apisix plugins configuration](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-736fc710c98e856e817ba0686d7549e499790399%2FapisixRoutesPluginsConfig.png?alt=media) For use by Griaule applications, some interesting plugins are: `limit-conn`, `limit-count`, `limit-req`, `traffic-split`, among others. Go to the [plugins page](https://apisix.apache.org/plugins/) for more details about each of the plugins available in APISIX. {% hint style="success" %} APISIX offers the possibility of developing custom plugins. [Click here](https://apisix.apache.org/docs/apisix/plugin-develop/) for more information. {% endhint %} {% hint style="info" %} In the Plugin tab of the Dashboard it is possible to enable a number of plugins for the API being configured. {% endhint %} {% hint style="info" %} Plugins enabled in the Plugin tab of the Dashboard are enabled globally for the API being configured. In tests, global plugins only worked when none of the routes had a plugin associated with them. {% endhint %} {% endstep %} {% endstepper %} #### Services and Consumers (Not used in tests) Through the Dashboard it is also possible to define Services and Consumers, which aim to simplify the configuration of plugins for certain routes. Instead of configuring plugins individually for each route, it is possible to configure a set of plugins in a Consumer, for example. Then, once APISIX identifies that specific Consumer (**through an authentication plugin**), the plugins configured for that Consumer begin to work on the requested route. Another alternative is to define an Upstream + Plugins through a Service. For this, use the Service and Consumer tabs of the Dashboard. {% hint style="success" %} For more details, visit the [Services documentation](https://apisix.apache.org/docs/apisix/terminology/service/) and the [Consumers documentation](https://apisix.apache.org/docs/apisix/terminology/consumer/). {% endhint %} By following these steps, it is possible to configure an API for APISIX. If you have any questions or problems with the Dashboard, check the [User Guide in the official APISIX Documentation](https://apisix.apache.org/docs/dashboard/USER_GUIDE/) or the help channels specified in the documentation. {% hint style="info" %} After configuring the routes, you need to publish them through the dashboard so they can be accessed correctly. {% endhint %} ### Configuring the API via command line/configuration APISIX allows configuring the application and its routes via the command line and configuration file, as detailed [here](https://apisix.apache.org/docs/apisix/getting-started/). For this, you need to specify the server (Upstream) and each of its routes, and follow the step-by-step below: #### Accessing the APISIX Admin ```sh curl "http://127.0.0.1:9080/apisix/admin/services/" -H 'X-API-KEY:edd1c9f034335f136f87ad84b625c8f1' ``` Expected response (indicates that APISIX is running correctly): ```json { "count": 0, "action": "get", "node": { "key": "/apisix/services", "nodes": [], "Dir": true } } ``` #### Creating a Route ```sh curl "http://127.0.0.1:9080/apisix/admin/routes/1" \ -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" \ -X PUT \ -d '{ "methods": ["GET"], "host": "example.com", "uri": "/anything/*", "upstream": { "type": "roundrobin", "nodes": { "httpbin.org:80": 1 } } }' ``` This configuration indicates that all incoming requests matching the Upstream service (`httpbin.org:80`) will be forwarded if they meet these specified criteria: * The HTTP method of the request is `GET`. * The request header contains the host field and its value is `example.com`. * The request path matches `/anything/*`, where `*` means any subpath. For example, `/anything/foo?arg=10`. With the Route created, you can access the Upstream service from the address exposed by APISIX: ```sh curl -i -X GET "http://127.0.0.1:9080/anything/foo?arg=10" -H "Host: example.com" ``` This request will be forwarded to `http://httpbin.org:80/anything/foo?arg=10` by APISIX. {% hint style="success" %} Instead of configuring the Upstream directly for the Route, you can create an Upstream object and use it in the Route. {% endhint %} #### Creating an Upstream ```sh curl "http://127.0.0.1:9080/apisix/admin/upstreams/1" \ -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" \ -X PUT \ -d '{ "type": "roundrobin", "nodes": { "httpbin.org:80": 1 } }' ``` {% hint style="info" %} This is the same as the Upstream service that was configured directly in the Route in the previous section. {% endhint %} #### Binding this Upstream to the Route You can use the `upstream_id` as `1`: ```sh curl "http://127.0.0.1:9080/apisix/admin/routes/1" \ -H "X-API-KEY: edd1c9f034335f136f87ad84b625c8f1" \ -X PUT \ -d '{ "methods": ["GET"], "host": "example.com", "uri": "/anything/*", "upstream_id": "1" }' ``` With the Route created, you can access the Upstream service from the address exposed by APISIX: ```sh curl -i -X GET "http://127.0.0.1:9080/anything/foo?arg=10" -H "Host: example.com" ``` This request will be forwarded to `http://httpbin.org:80/anything/foo?arg=10` by APISIX. ## Importing an API exported by APISIX To facilitate API configuration, along with product releases, a file is made available `.yaml` generated by APISIX with the routes used by each of the web applications. With this, it is possible to import the routes directly through the Dashboard, speeding up the configuration process on the bus. However, APISIX's import/export functionality has some limitations and, therefore, additional steps are required after import so that the bus works as expected. {% hint style="info" %} To facilitate exporting the files, the parameter validation plugin was removed from the routes before their export. Validation is performed by the servers. {% endhint %} ### Create an Upstream When exporting the application's routes through APISIX, the Upstream of each route (Backend API Server) is exported as if it had been specified for each of the routes. This behavior makes API configuration difficult if there are changes in the application's server address, machine changes, etc. Therefore, it is recommended to create an Upstream related to the application server being configured on the bus to centralize the server and facilitate possible changes. {% hint style="info" %} When creating the Upstream, pay attention to the timeouts specified for the server (it is possible to configure connection, send and receive timeout). On slow servers, if the request takes longer than the specified timeout to return, a *504 Error* will be returned by APISIX itself. {% endhint %} {% hint style="info" %} By default, the exported routes will point to their respective server running at IP `172.16.0.66` (development machine). {% endhint %} ### Configuring the Routes As mentioned earlier, the functionality to export an API's configurations through APISIX has some flaws and, therefore, to fully configure the API on the bus, some changes must be made to all routes that were imported from the generated .yaml file. Although they are specified in the file generated by APISIX, calls with the HTTP request type `OPTIONS` are not converted for the routes imported into APISIX. For this reason, to allow web applications to access the routes, it is necessary to modify route by route and add the OPTIONS method as one of the possible methods so that requests work as expected, avoiding CORS errors. {% hint style="info" %} Adding the OPTIONS method is the only modification necessary in the Define API Request tab. {% endhint %} Next, in the Define API Backend Server tab, you must change the server (using the dropdown menu) and select the Upstream related to the application server being configured - which should have been created previously. In the Plugins tab, only plugins that are specific to the route being configured should be added. Otherwise, you can enable plugins in the Plugin tab of the Dashboard. Plugins enabled in the Plugin tab are enabled globally and will work for all routes configured in APISIX. Thus, for each of the imported routes it is necessary to: 1. Add OPTIONS method 2. Select the Upstream 3. Configure plugin (only if it is specific to the route being configured) 4. Publish By following these steps, it is possible to import the API exported through APISIX and use the bus features to access the application. ### Accessing the Routes After configuring all application routes and publishing them, to access them through the web application, you need to configure the application to make calls through the bus. If you are running APISIX on your machine, just configure the route to: `http://127.0.0.1:9080//service` Or to the port `9080` of the machine running APISIX. ## FAQ For more specifications, step-by-step or questions, you can consult the [Official Documentation](https://apisix.apache.org/docs/apisix/getting-started/) or access APISIX support channels. ### Error importing OpenAPI file #### APISIX reports a formatting error in the file When trying to import the OpenAPI file through the APISIX Dashboard, if the generated file is not correctly formatted, an error toast will be shown on the screen. To validate whether the file is a valid OpenAPI, use the [Swagger Editor](https://editor.swagger.io) (which reports the error or unused parameters) or another validation tool. #### APISIX Dashboard crashes and logs out the user after importing file During import tests through the Dashboard, 2 situations were found that led to this error. Both were related to an unidentified problem in schemas declared in the `swagger.yaml`. A practical example, found when importing the ETR, was identified in the endpoint `/commons/version` in the schema `SystemConfiguration` declared as a bodyRequest parameter of the endpoint: ```yml # ... post: summary: Set system configuration operationId: set parameters: - name: session-guid in: header schema: type: string requestBody: content: '*/*': schema: $ref: '#/components/schemas/SystemConfiguration' required: false # ... ``` On that occasion, this schema was removed from the OpenAPI file and the import occurred without issues. After investigation, it was found that the exported schema was out of date compared to what the server expected but, even after the fix, the error persisted. In this case, the alternative is to remove the problematic schema and perform the import again. By doing this, the schema will not be mapped to the `request-validation` plugin and a request with an incorrect parameter will pass through APISIX. However, this ends up not being a problem because an incorrect parameter will cause an error on the server itself, which will be returned to the user, mitigating the absence of the schema in the imported file. In the case of the `BEST`, for example, the adopted alternative was **replace all** `requestBody` **attributes with an empty object (**`{}`**) and add some parameter for validation in the** `header` - this is only valid if parameter validation is disabled for the routes. Otherwise, it will cause a Bad request error from APISIX when receiving the request. Another alternative to try to solve the problem is to check APISIX's question channels (Slack, GitHub and blogs) for definitive solutions to this problem. **Although it was found that this problem is related to the** `schemas` **data**. ### declared in our servers, a definitive solution was not found Plugin configured on the route is not working **By default, when adding a plugin to the Route or to the Consumer, the plugin is marked as**disabled `Enabled` and ensure it is active so that the plugin works correctly on the route. ![apisix plugin editor screen](https://3757157672-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F7Bx0xNdsdGHpCZ20yxbn%2Fuploads%2Fgit-blob-f8f826bfd1c59152e011517d2ed05cfee72b33c5%2FapisixPluginEditor.png?alt=media) ### Unable to access the APISIX bus after running it During tests of the bus using APISIX, it was found that routes were not being correctly recognized/redirected by APISIX, even though it was running correctly via Docker. After investigating the problem, it was found that there was an application running on port `9080` of the machine, which is the port used by the APISIX API. This causes the APISIX API to be inaccessible and the routes are not found when making a request. The solution is to stop the process running on the target port, or change APISIX's default port through the configuration file.