[influxdb] Using nfs for influxdb/db Storage

[e***@gmail.com]

I was running out of space on my /var filesystem so I thought I would use an nfs mount:

/etc/fstab
nfs-server01:/influxdb /var/opt/influxdb/db nfs rw,intr,soft 0 0

After I uninstalled, re-installed and re-started influxdb I tried to create a new database:

$ /opt/influxdb/influx
Connected to http://localhost:8086 version
InfluxDB shell 0.9.0-rc27

show databases

ERR: invalid character 'o' in literal null (expecting 'u')

create database mydb

ERR: invalid character 'o' in literal null (expecting 'u')

show databases

ERR: invalid character 'o' in literal null (expecting 'u')

exit

I also tried uninstalling and removing everything else from under /var/opt/influxdb as well as /opt/influxdb with the same results. I tried rc27 and rc28.

After removing the mount and doing another clean install everything was working again.

regards,

Simon.

[r***@richardelling.com]

NFS may present challenges for InfluxDB right now: https://groups.google.com/d/msgid/influxdb/05e881d2-684b-4d5b-a9ae-8fd11e0aac8d%40googlegroups.com. We have not started meaningful testing across different filesystems but I've added this to the test plan.

I've been running 0.8.8 over NFS for some time with no issues. I'd be happy to
help debug.
-- richard

[p***@gmail.com]

Post by r***@richardelling.com

NFS may present challenges for InfluxDB right now: https://groups.google.com/d/msgid/influxdb/05e881d2-684b-4d5b-a9ae-8fd11e0aac8d%40googlegroups.com. We have not started meaningful testing across different filesystems but I've added this to the test plan.

I've been running 0.8.8 over NFS for some time with no issues. I'd be happy to
help debug.
-- richard

Hi Richard,

I have installed Influx db in Openshift container and would like to mount AWS NFS to this Influx data. After mounting NFS volume to the influx container, i am getting below error while instantiating container.

run: create server: mkdir all: mkdir /var/lib/influxdb/meta: permission denied

I have given full access to the NFS file system.
Can you let me know what is the issue here?

Regards,
Madhu

[Richard Elling]

Hi Madhu,
The first question I have is whether you're using NFS version 3 or 4. The
answer differs for each. In NFS version 3 we see such issues more
frequently. In NFS version 3 the "owner" is a number, not a string. So the
mapping of userid (number) to username (string) works best when all of the
NFS clients agree. For example, the influxdb userid (number) is hopefully
the same everywhere in your system.

Note: there is special handling of userid 0 (typically username=root) that
has configuration options on both the client and server to map to a
different userid (typically username=nobody).

For such issues, "ls -ln" is an easy way to verify the files agree with the
username mapping.

NFSv4 uses actual strings, where we have fewer of these issues.
-- richard

[p***@gmail.com]

Post by Richard Elling
Hi Madhu,
The first question I have is whether you're using NFS version 3 or 4. The answer differs for each. In NFS version 3 we see such issues more frequently. In NFS version 3 the "owner" is a number, not a string. So the mapping of userid (number) to username (string) works best when all of the NFS clients agree. For example, the influxdb userid (number) is hopefully the same everywhere in your system.
Note: there is special handling of userid 0 (typically username=root) that has configuration options on both the client and server to map to a different userid (typically username=nobody).
For such issues, "ls -ln" is an easy way to verify the files agree with the username mapping.
NFSv4 uses actual strings, where we have fewer of these issues.
 -- richard

Hi Richard,

I don't know the version of NFS which is used in AWS EFS system.
When I checked ls -ln on mounted folders, i see below for influx-data folder.
I am able to use prometheus-data folder to store Promethues logs. But it is not working for influxdb container. Let me know any thoughts on this.

[***@ip-10-100-XX-XX shared-data]$ ls -ln
total 20
drwxr-xr-x. 4 65534 65534 6144 Jul 28 09:48 exports
drwxrwxrwx. 2 0 0 6144 Oct 5 09:40 influx-data
drwxrwxrwx. 262 0 0 14336 Oct 6 14:32 prometheus-data

[***@ip-10-100-xx-xx shared-data]$ ls -lrt
total 20
drwxr-xr-x. 4 nfsnobody nfsnobody 6144 Jul 28 09:48 exports
drwxrwxrwx. 2 root root 6144 Oct 5 09:40 influx-data
drwxrwxrwx. 262 root root 14336 Oct 6 14:32 prometheus-data
You have new mail in /var/spool/mail/centos

Regards,
Madhu

[Richard Elling]

Hi Madhu,

Post by Richard Elling

Post by Richard Elling
Hi Madhu,
The first question I have is whether you're using NFS version 3 or 4.

The answer differs for each. In NFS version 3 we see such issues more
frequently. In NFS version 3 the "owner" is a number, not a string. So the
mapping of userid (number) to username (string) works best when all of the
NFS clients agree. For example, the influxdb userid (number) is hopefully
the same everywhere in your system.

Post by Richard Elling
Note: there is special handling of userid 0 (typically username=root)

that has configuration options on both the client and server to map to a
different userid (typically username=nobody).

Post by Richard Elling
For such issues, "ls -ln" is an easy way to verify the files agree with

the username mapping.

Post by Richard Elling
NFSv4 uses actual strings, where we have fewer of these issues.
-- richard

Hi Richard,
I don't know the version of NFS which is used in AWS EFS system.
When I checked ls -ln on mounted folders, i see below for influx-data folder.
I am able to use prometheus-data folder to store Promethues logs. But it
is not working for influxdb container. Let me know any thoughts on this.
total 20
drwxr-xr-x. 4 65534 65534 6144 Jul 28 09:48 exports
drwxrwxrwx. 2 0 0 6144 Oct 5 09:40 influx-data
drwxrwxrwx. 262 0 0 14336 Oct 6 14:32 prometheus-data
total 20
drwxr-xr-x. 4 nfsnobody nfsnobody 6144 Jul 28 09:48 exports
drwxrwxrwx. 2 root root 6144 Oct 5 09:40 influx-data
drwxrwxrwx. 262 root root 14336 Oct 6 14:32 prometheus-data
You have new mail in /var/spool/mail/centos

The user "nfsnobody" is an indication that the root user on the client is
mapped
to the non-root user nfsnobody. The best solution to this is to not install
or run
influxdb as root. For example, I notice that the rpms distributed on the
influxdata.com
website install a user called "influxdb" and do not run as root, by
default. This is a
good idea.
-- richard

Post by Richard Elling
Regards,
Madhu