[influxdb] Influxdb using up a hell lot of RAM

[sarad mohanan]

I have a single instance influxdb installation on AWS. Initally I installed
it on a t2.small maching (2GB ram and 1 CPU) about 5 weeks ago. Also I have
some 10-15 other machines around with telegraf reporting about some vital
system metrics and some application metrics initially everything was fine
within 3-4 days it started up using more RAM so I changed instance type to
t2.medium and then again everything became normal not long after that I had
to change to t2.large and 2 days back I changed it to t2.xlarge! a machine
with 16GB of RAM and current RAM usage is around 12GB :O. I have a total
data of 2.5GB on disk also I have put in retention period(RP) of around 3
weeks max(I have some RPs of 1 week). What might be the problem here.

[Frank Inselbuch]

What operating system?
What process(es) are using a lot of memory?
Please clarify "it started up using more RAM" ...
Is there anything else running on the machine?

Post by sarad mohanan
I have a single instance influxdb installation on AWS. Initally I
installed it on a t2.small maching (2GB ram and 1 CPU) about 5 weeks ago.
Also I have some 10-15 other machines around with telegraf reporting about
some vital system metrics and some application metrics initially everything
was fine within 3-4 days it started up using more RAM so I changed instance
type to t2.medium and then again everything became normal not long after
that I had to change to t2.large and 2 days back I changed it to t2.xlarge!
a machine with 16GB of RAM and current RAM usage is around 12GB :O. I have
a total data of 2.5GB on disk also I have put in retention period(RP) of
around 3 weeks max(I have some RPs of 1 week). What might be the problem
here.

[sarad mohanan]

@Frank

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

- Influxdb is using up all the RAM!
- as the data started flowing in from telegraf influxdb started up using
more ram
- no, its just a dedicated machine for influx

Post by sarad mohanan
I have a single instance influxdb installation on AWS. Initally I
installed it on a t2.small maching (2GB ram and 1 CPU) about 5 weeks ago.
Also I have some 10-15 other machines around with telegraf reporting about
some vital system metrics and some application metrics initially everything
was fine within 3-4 days it started up using more RAM so I changed instance
type to t2.medium and then again everything became normal not long after
that I had to change to t2.large and 2 days back I changed it to t2.xlarge!
a machine with 16GB of RAM and current RAM usage is around 12GB :O. I have
a total data of 2.5GB on disk also I have put in retention period(RP) of
around 3 weeks max(I have some RPs of 1 week). What might be the problem
here.

[Frank Inselbuch]

can you capture the output from

top

Sent from my iPad

On Jul 12, 2017, at 2:29 AM, sarad mohanan <***@gmail.com<mailto:***@gmail.com>> wrote:

@Frank

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

- Influxdb is using up all the RAM!
- as the data started flowing in from telegraf influxdb started up using more ram
- no, its just a dedicated machine for influx


On Monday, 10 July 2017 17:03:43 UTC+5:30, sarad mohanan wrote:
I have a single instance influxdb installation on AWS. Initally I installed it on a t2.small maching (2GB ram and 1 CPU) about 5 weeks ago. Also I have some 10-15 other machines around with telegraf reporting about some vital system metrics and some application metrics initially everything was fine within 3-4 days it started up using more RAM so I changed instance type to t2.medium and then again everything became normal not long after that I had to change to t2.large and 2 days back I changed it to t2.xlarge! a machine with 16GB of RAM and current RAM usage is around 12GB :O. I have a total data of 2.5GB on disk also I have put in retention period(RP) of around 3 weeks max(I have some RPs of 1 week). What might be the problem here.

--
Remember to include the version number!
---
You received this message because you are subscribed to a topic in the Google Groups "InfluxData" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/influxdb/Z6WDmK4Xb1o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to influxdb+***@googlegroups.com<mailto:influxdb+***@googlegroups.com>.
To post to this group, send email to ***@googlegroups.com<mailto:***@googlegroups.com>.
Visit this group at https://groups.google.com/group/influxdb.
To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/22cdac14-de96-4a2e-a21c-38d60c5dc384%40googlegroups.com<https://groups.google.com/d/msgid/influxdb/22cdac14-de96-4a2e-a21c-38d60c5dc384%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

[Peter Booth]

Do you have a lot of continuous queries defined? Are you mostly writing or reading ? Are the request times getting slower? Are you overwriting the same datapoint sir writing out of time order? When you look at top for a few minutes do you see the load average consistently above 50%!

[sarad mohanan]

here is the output of top

top - 11:14:22 up 2 days, 23:58, 1 user, load average: 0.36, 0.16, 0.11
Tasks: 133 total, 1 running, 132 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.3 us, 0.2 sy, 0.0 ni, 98.4 id, 1.0 wa, 0.0 hi, 0.1 si,
0.1 st
KiB Mem : 16431212 total, 5864760 free, 6320760 used, 4245692 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 9785720 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16252 influxdb 20 0 12.485g 8.211g 2.350g S 2.0 52.4 339:39.80 influxd
1031 consul 20 0 34988 20212 8756 S 0.7 0.1 18:42.81 consul
311 root 20 0 0 0 0 S 0.3 0.0 0:44.72
jbd2/xvda1-8
384 root 20 0 73344 37792 29184 S 0.3 0.2 2:53.75
systemd-journal
1212 www-data 20 0 127204 7876 5388 S 0.3 0.0 0:55.03 nginx
1213 www-data 20 0 127212 7608 5144 S 0.3 0.0 0:48.37 nginx
1 root 20 0 185140 5568 3792 S 0.0 0.0 0:02.94 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.38
ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00
kworker/0:0H
7 root 20 0 0 0 0 S 0.0 0.0 0:18.42
rcu_sched

[sarad mohanan]

Post by Peter Booth
Do you have a lot of continuous queries defined?

no, I have no continuous queries.

Post by Peter Booth
Are you mostly writing or reading?

mostly writing

Post by Peter Booth
Are the request times getting slower?

No

Post by Peter Booth
Are you overwriting the same datapoint sir writing out of time order?

NO!

Post by Peter Booth
When you look at top for a few minutes do you see the load average

consistently above 50%!
Load average is rarely above 0.1

Post by Peter Booth
I have a single instance influxdb installation on AWS. Initally I
installed it on a t2.small maching (2GB ram and 1 CPU) about 5 weeks ago.
Also I have some 10-15 other machines around with telegraf reporting about
some vital system metrics and some application metrics initially everything
was fine within 3-4 days it started up using more RAM so I changed instance
type to t2.medium and then again everything became normal not long after
that I had to change to t2.large and 2 days back I changed it to t2.xlarge!
a machine with 16GB of RAM and current RAM usage is around 12GB :O. I have
a total data of 2.5GB on disk also I have put in retention period(RP) of
around 3 weeks max(I have some RPs of 1 week). What might be the problem
here.