Cómo migrar de InfluxDB OSS 2.x a InfluxDB Cloud usando Python

En este artículo te muestro como exportar datos desde InfluxDB OSS 2.x a InfluxDB Cloud usando scripting en Python.


8 min de lectura
Cómo migrar de InfluxDB OSS 2.x a InfluxDB Cloud usando Python


Si sos un usuario de InfluxDB OSS, que lo usa para cargas productivas, por ahí llegarás al momento en que te quedé chico y ahí el camino son dos, uno, comprar una licencia enterprise y armar un cluster on-premise o en la nube o migrar tu data y escalar en función de tus necesidad en InfluxDB Cloud.

En este artículo, enfocado en la segunda opción, veremos como migrar tu data de InfluxDB OSS 2.x a InfluxDB Cloud con un script en Python que desarrollé con esta idea.

Empecemos

En este repo de Github encontrarán los archivos necesarios para comenzar a exportar la data desde InfluxDB OSS 2.x a InfluxDB Cloud.

xe-nvdk/influxdb-cloud-importer
This Python script allows you to export data from a bucket in InfluxDB OSS 2.x to sent to InfluxDB Cloud Instance. - xe-nvdk/influxdb-cloud-importer

Primero, lo que debemos es instalar el cliente Python para InfluxDB:

$ pip3 install influxdb_client

Una vez instalado, lo siguiente es clonar el repositorio donde están guardado los archivos que necesitamos:

$ git clone https://github.com/xe-nvdk/influxdb-cloud-importer.git

Una vez descargado, entramos a la carpeta y nos encontraremos con algunos archivos, los más relevantes son main.py y values.py. El primero es el que vamos a correr y hablará con cada script en función de las opciones que elijamos, pero el segundo, es donde fijaremos los valores de nuestra instancia Open Source y Cloud. Veamos.

# Define the source
src_instance = "" # Enter your InfluxDB URL (Ex: localhost:8086).
src_bucket = "" # Enter the bucket that from you want to download data.
src_org = "" # Enter your organisation.
src_token = "" # Enter your token, make sure that the token has permission to read the bucket.
src_time = "-2h" # Define how much data you want to export. Can be -1m, -6h, -2d, -30d, etc.

# define the destination
dst_instance = "" # Enter your InfluxDB Cloud instace. (Ex: https://us-west-2-1.aws.cloud2.influxdata.com).
dst_bucket = "" # Enter the destination bucket. Needs to be created previuosly.
dst_org = "" # Enter your organisation, this can be your org id or the email address used to register to InfluxDB Cloud.
dst_token = "" # Enter your destination token, make sure that has permission to write in the bucket.

Lo importante aquí es que debemos definir el nombre del bucket en donde esta nuestra información y la cantidad de datos en tiempo que vamos a extraer. Con respecto a este último dato, hay que ser conscientes de lo que necesitamos, ya que uno de las variables para el pricing de InfluxDB Cloud es la data que enviamos + el storage que ocupamos.

Si, por ejemplo, necesitamos 30 días, en donde dice "src_time" especificaremos "-30d".

Una vez definido el bucket, token, instancia, organización y el tiempo que queremos exportar, vamos a la sección de destination.

Aquí debemos definir el bucket a donde vamos a volcar nuestra data, así como también la instancia, un token, que tenga permisos para escribir a ese bucket, la organización y nada más.

Una vez que tenemos los valores definidos, podemos empezar a exportar.

Exportemos

Con esta tool podemos exportar dos cosas, una, va a ser los dashboards, tareas, buckets, configuraciones de Telegraf, etiquetas y creo que nada más, pero también podemos exportar datos o hacer las dos cosas al mismo tiempo.

Veamos:

$ python3 main.py
#######################################################################################################################
########################### Welcome to InfluxDB Cloud Importer by Ignacio Van Droogenbroeck ###########################
#######################################################################################################################

I'm going to help you to export your entire stack from InfluxDB Cloud OSS to InfluxDB Cloud

What do you want to do?

Press 1 to export Dashboards, Labels, Telegraf configuration, alerts, etc.
Press 2 to export data.
Press 99 to do both.

Enter your choice:

Acá lo que debemos definir es que si apretamos "1", va a exportar todos los dashboards, labels y demás como comenté más arriba, si apretamos "2", va a exportar data y si introducimos "99", hará las dos cosas.

Vamos con la primera opción y exportemos algunos dashboards, introducimos "1" y le damos enter... lo siguiente que pasará será algo parecido a esto:

I'm exporting your resources...
It's done

Now, I'm importing this to your InfluxDB Cloud instance. You need to confirm...
LABELS    +add | -remove | unchanged
+-----+------------------------------+------------------+-----------------------+---------+-------------+
| +/- |        METADATA NAME         |        ID        |     RESOURCE NAME     |  COLOR  | DESCRIPTION |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | agreeing-bartik-80300b       | 074f68a47d2a7000 | experimental          | #BF3D5E |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | beautiful-matsumoto-403011   | 0722670a30759000 | inputs.net            | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | compassionate-lalande-403005 | 0722670a27b59000 | inputs.disk           | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | dazzling-almeida-c03005      | 074f68a47a2a7000 | gaming                | #dd84f1 |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | endangered-robinson-403013   | 0722670a29759000 | inputs.processes      | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | gallant-shaw-803005          | 074f68a4796a7000 | Linux                 | #009f5f |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | gracious-goldstine-803001    | 074f68a47dea7000 | DigitalOcean          | #066fc5 |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | happy-chaum-403015           | 0722670a22b59000 | inputs.swap           | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | hopeful-galileo-40300f       | 0722670a2cb59000 | inputs.mem            | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | hopeful-saha-803009          | 06c7221c2c0e4000 | covid                 | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | jovial-hugle-403003          | 0722670a2ef59000 | inputs.cpu            | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | kind-wright-803007           | 0722670a34f59000 | Linux System Template | #7A65F2 |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | laughing-shannon-40300b      | 074f68a47b6a7000 | inputs.kube_inventory | #066fc5 |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | musing-driscoll-403007       | 0722670a20759000 | inputs.diskio         | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | nice-easley-403019           | 0722670a23f59000 | outputs.influxdb_v2   | #108174 |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | objective-williams-c03003    | 074f68a47e6a7000 | fortnite              | #00a3ff |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | peaceful-cannon-403009       | 0722670a2ab59000 | inputs.kernel         | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | relaxed-gates-803003         | 074f68a47aea7000 | K8S                   | #757888 |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | ridiculous-germain-40300d    | 074f68a47baa7000 | inputs.kubernetes     | #00a3ff |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | wonderful-pare-403017        | 0722670a31b59000 | inputs.system         | #326BBA |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | wondrous-herschel-c03001     | 074f68a47f2a7000 | fail2ban              | #67d74e |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|     | zen-lederberg-403001         | 074f68a47c2a7000 | github                | #00a3ff |             |
+-----+------------------------------+------------------+-----------------------+---------+-------------+
|                                                                                  TOTAL  |     22      |
+-----+------------------------------+------------------+-----------------------+---------+-------------+

BUCKETS    +add | -remove | unchanged
+-----+----------------------------+------------------+-------------------+------------------+-------------+
| +/- |       METADATA NAME        |        ID        |   RESOURCE NAME   | RETENTION PERIOD | DESCRIPTION |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|     | amazing-driscoll-803005    | 38f0e3e744b18055 | env-fn_bucket     | 720h0m0s         |             |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|     | crumbling-almeida-803007   | 88bbf2e113d93bf3 | mssql             | 0s               |             |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|     | determined-fermat-80300b   | cecb9cb2d285b2aa | system_monitoring | 336h0m0s         |             |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|     | modest-swartz-80300d       | 2de0bd1516670b61 | watcher           | 0s               |             |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|     | pensive-ardinghelli-803001 | 29acb10e1026c604 | covid             | 0s               |             |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|     | practical-matsumoto-803009 | 4b6a833c119ea6b7 | public-count      | 0s               |             |
+-----+----------------------------+------------------+-------------------+------------------+-------------+
|                                                                                TOTAL       |      6      |
+-----+----------------------------+------------------+-------------------+------------------+-------------+

CHECKS    +add | -remove | unchanged
+-----+---------------------------+------------------+---------------+-------------+
| +/- |       METADATA NAME       |        ID        | RESOURCE NAME | DESCRIPTION |
+-----+---------------------------+------------------+---------------+-------------+
|     | hardcore-zhukovsky-c03001 | 074f68a482e63000 | memory km-usa |             |
+-----+---------------------------+------------------+---------------+-------------+
|                                                          TOTAL     |      1      |
+-----+---------------------------+------------------+---------------+-------------+

DASHBOARDS    +add | -remove | unchanged
+-----+--------------------------+----+-----------------+--------------------------------+------------+
| +/- |      METADATA NAME       | ID |  RESOURCE NAME  |          DESCRIPTION           | NUM CHARTS |
+-----+--------------------------+----+-----------------+--------------------------------+------------+
| +   | gallant-beaver-003001    |    | Kube Cluster DO | This dashboard is dedicated    | 12         |
|     |                          |    |                 | for monitoring K3s Cluster in  |            |
|     |                          |    |                 | DigitalOcean                   |            |
+-----+--------------------------+----+-----------------+--------------------------------+------------+
+-----+--------------------------+----+-----------------+--------------------------------+------------+
| +   | objective-taussig-803001 |    | Cars            |                                | 1          |
+-----+--------------------------+----+-----------------+--------------------------------+------------+
+-----+--------------------------+----+-----------------+--------------------------------+------------+
| +   | pedantic-hawking-403001  |    | Fornite Stats   |                                | 8          |
+-----+--------------------------+----+-----------------+--------------------------------+------------+
+-----+--------------------------+----+-----------------+--------------------------------+------------+
| +   | tasty-ramanujan-403001   |    | COVID19         |                                | 57         |
+-----+--------------------------+----+-----------------+--------------------------------+------------+
|                                                                     TOTAL              |     4      |
+-----+--------------------------+----+-----------------+--------------------------------+------------+

TELEGRAF CONFIGURATIONS    +add | -remove | unchanged
+-----+------------------------+----+--------------------------+-------------+
| +/- |     METADATA NAME      | ID |      RESOURCE NAME       | DESCRIPTION |
+-----+------------------------+----+--------------------------+-------------+
| +   | pedantic-cerf-403007   |    | K8S Configuration        |             |
+-----+------------------------+----+--------------------------+-------------+
+-----+------------------------+----+--------------------------+-------------+
| +   | zealous-wescoff-803001 |    | Fortnite Telegraf config |             |
+-----+------------------------+----+--------------------------+-------------+
|                                              TOTAL           |      2      |
+-----+------------------------+----+--------------------------+-------------+

VARIABLES    +add | -remove | unchanged
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
| +/- |         METADATA NAME         |        ID        | RESOURCE NAME | DESCRIPTION | ARG TYPE |                         ARG VALUES                          |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
|     | dazzling-chandrasekhar-003017 | 074f68a47fded000 | jail          |             | query    | language="flux" query="import                               |
|     |                               |                  |               |             |          | \"influxdata/influxdb/v1\"\n\nall_data                      |
|     |                               |                  |               |             |          | = buckets() |> limit(n:1) |>                                |
|     |                               |                  |               |             |          | set(key: \"_value\", value: \"*\")                          |
|     |                               |                  |               |             |          | |> keep(columns:[\"_value\"])\njails                        |
|     |                               |                  |               |             |          | = v1.measurementTagValues(bucket:                           |
|     |                               |                  |               |             |          | v.bucket, measurement: \"fail2ban\",                        |
|     |                               |                  |               |             |          | tag: \"jail\")\n\nunion(tables:                             |
|     |                               |                  |               |             |          | [all_data, jails]) |> sort()"                               |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
|     | inspiring-lewin-00300b        | 074f68a4801ed000 | host          |             | query    | language="flux" query="import                               |
|     |                               |                  |               |             |          | \"influxdata/influxdb/v1\"\n\nall_data                      |
|     |                               |                  |               |             |          | = buckets() |> limit(n:1) |>                                |
|     |                               |                  |               |             |          | set(key: \"_value\", value: \"*\")                          |
|     |                               |                  |               |             |          | |> keep(columns:[\"_value\"])\nhosts                        |
|     |                               |                  |               |             |          | = v1.measurementTagValues(bucket:                           |
|     |                               |                  |               |             |          | v.bucket, measurement: \"fail2ban\",                        |
|     |                               |                  |               |             |          | tag: \"host\")\n\nunion(tables:                             |
|     |                               |                  |               |             |          | [all_data, hosts]) |> sort()"                               |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
|     | reverent-napier-003007        | 0722670a3661b000 | bucket        |             | query    | language="flux"                                             |
|     |                               |                  |               |             |          | query="buckets()\n  |>                                      |
|     |                               |                  |               |             |          | filter(fn: (r) => r.name !~                                 |
|     |                               |                  |               |             |          | /^_/)\n  |> rename(columns:                                 |
|     |                               |                  |               |             |          | {name: \"_value\"})\n  |>                                   |
|     |                               |                  |               |             |          | keep(columns: [\"_value\"])"                                |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
|     | shiny-brahmagupta-00300f      | 074f68a4809ed000 | season        |             | query    | language="flux" query="import                               |
|     |                               |                  |               |             |          | \"influxdata/influxdb/v1\"\n\nv1.tagValues(\n               |
|     |                               |                  |               |             |          |  bucket: \"fortnite\",\n  tag: \"season\",\n                |
|     |                               |                  |               |             |          | predicate: (r) => true,\n  start: -300d\n)"                 |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
|     | sleepy-tharp-003013           | 0722670a3a61b000 | linux_host    |             | query    | language="flux" query="import                               |
|     |                               |                  |               |             |          | \"influxdata/influxdb/v1\"\nv1.measurementTagValues(bucket: |
|     |                               |                  |               |             |          | v.bucket, measurement: \"cpu\", tag: \"host\")"             |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+
|                                                                                         TOTAL   |                              5                              |
+-----+-------------------------------+------------------+---------------+-------------+----------+-------------------------------------------------------------+

LABEL ASSOCIATIONS    +add | -remove | unchanged
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +/- | RESOURCE TYPE |      RESOURCE META NAME       |   RESOURCE NAME   |   RESOURCE ID    |    LABEL PACKAGE NAME     |      LABEL NAME       |     LABEL ID     |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | buckets       | amazing-driscoll-803005       | env-fn_bucket     | 38f0e3e744b18055 | objective-williams-c03003 | fortnite              | 074f68a47e6a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | buckets       | pensive-ardinghelli-803001    | covid             | 29acb10e1026c604 | hopeful-saha-803009       | covid                 | 06c7221c2c0e4000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | dashboards    | gallant-beaver-003001         | Kube Cluster DO   |                  | gracious-goldstine-803001 | DigitalOcean          | 074f68a47dea7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | dashboards    | gallant-beaver-003001         | Kube Cluster DO   |                  | gallant-shaw-803005       | Linux                 | 074f68a4796a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | dashboards    | objective-taussig-803001      | Cars              |                  | agreeing-bartik-80300b    | experimental          | 074f68a47d2a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | dashboards    | pedantic-hawking-403001       | Fornite Stats     |                  | dazzling-almeida-c03005   | gaming                | 074f68a47a2a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | dashboards    | tasty-ramanujan-403001        | COVID19           |                  | hopeful-saha-803009       | covid                 | 06c7221c2c0e4000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | telegrafs     | pedantic-cerf-403007          | K8S Configuration |                  | laughing-shannon-40300b   | inputs.kube_inventory | 074f68a47b6a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
| +   | telegrafs     | pedantic-cerf-403007          | K8S Configuration |                  | ridiculous-germain-40300d | inputs.kubernetes     | 074f68a47baa7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | dazzling-chandrasekhar-003017 | jail              | 074f68a47fded000 | wondrous-herschel-c03001  | fail2ban              | 074f68a47f2a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | inspiring-lewin-00300b        | host              | 074f68a4801ed000 | wondrous-herschel-c03001  | fail2ban              | 074f68a47f2a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | reverent-napier-003007        | bucket            | 0722670a3661b000 | kind-wright-803007        | Linux System Template | 0722670a34f59000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | reverent-napier-003007        | bucket            | 0722670a3661b000 | wondrous-herschel-c03001  | fail2ban              | 074f68a47f2a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | reverent-napier-003007        | bucket            | 0722670a3661b000 | objective-williams-c03003 | fortnite              | 074f68a47e6a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | shiny-brahmagupta-00300f      | season            | 074f68a4809ed000 | objective-williams-c03003 | fortnite              | 074f68a47e6a7000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|     | variables     | sleepy-tharp-003013           | linux_host        | 0722670a3a61b000 | kind-wright-803007        | Linux System Template | 0722670a34f59000 |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+
|                                                                                                                                  TOTAL         |        16        |
+-----+---------------+-------------------------------+-------------------+------------------+---------------------------+-----------------------+------------------+

Confirm application of the above resources (y/n): y

Como pueden ver, la cantidad de datos es importante y es por eso que hay mucho contenido en el snippet de aquí arriba, pero si estamos seguro, introducimos "y" y le damos enter...

Lo que veremos al finalizar será una confirmación de que se pudo hacer todo como se esperaba:

stack ID: 075eced136149000
You're ready to roll

Ahora probemos exportando data. Para esto, vamos a usar la opción 2.

Corremos de nuevo el archivo main.py.

$ python3 main.py
#######################################################################################################################
########################### Welcome to InfluxDB Cloud Importer by Ignacio Van Droogenbroeck ###########################
#######################################################################################################################

I'm going to help you to export your entire stack from InfluxDB Cloud OSS to InfluxDB Cloud

What do you want to do?

Press 1 to export Dashboards, Labels, Telegraf configuration, alerts, etc.
Press 2 to export data.
Press 99 to do both.

Enter your choice: 2

En mi caso, va a comenzar a sacar los datos de mi instancia Open Source y los volcará en un archivo llamado exported.csv. Ahora bien, una cosa que agregue a este script es que haga un calculo del costo del data-in, si es que sos un usuario de PAYG de InfluxDB Cloud, ese será el cargo, si tu instancia en gratuita, no tendrás ningún cargo, ya que dependiendo de la cantidad de data estarás dentro del tier gratuito de uso.

Downloading bucket system_monitoring to exported.csv
If you're a PAYG user this write is going to cost you $ 0.009483207702636718
Do you want to continue: y/n: y

Si estamos seguros y aceptamos eso, introducimos "y" y le damos enter, veremos algo como esto:

Uploading the data, this can take some time depending of the size of your file: 4.74 MB

Una vez que este todo ok, el output será:

It's done, go to your instance in Cloud, log in, and explore the bucket, you should the see the information there.

A partir de ahí, podremos ir a nuestra instancia de Cloud, buscar el bucket hacia donde exportamos la información y deberíamos ver data:

Para ir cerrando

Este proyecto es parte de mi aprendizaje con Python y una respuesta a una consulta recurrente que veo en el Slack de InfluxDB que es, cómo exportar datos de InfluxDB OSS 2.x hacia InfluxDB Cloud. Seguramente hay mucho que mejorar así que te invito a contribuir a este proyecto en este enlace:

xe-nvdk/influxdb-cloud-importer
This Python script allows you to export data from a bucket in InfluxDB OSS 2.x to sent to InfluxDB Cloud Instance. - xe-nvdk/influxdb-cloud-importer

Espero que este artículo y esta solución te sirva. Si lo probaste y tuviste algún problema, por favor, no dudes en consultarme aquí abajo en los comentarios o través de las redes sociales.


baehost

SUBIR

🎉 Te has suscrito con éxito a CDUser - El Blog de Ignacio Van Droogenbroeck!
OK