Two ISC dhcpd servers running as a failover pair running Version 4.3.3-P1 compiled, running on Ubuntu 14.04 64 bit
The Servers are 500 gigs of RAID 1 space with 8 gigs of RAM with quad core Intel(R) Xeon(R) CPU E31225 @ 3.10GHz
The dhcpd.leases file sits in /ramdisk which is a 4 Gb RAM disk to make sure that the file can be written to very quickly, this method gives us virtually a 0.0 WA time when viewing with the top command.
The LEASE_HASH size was increased to 1800017 and enabled debugging with REPORT_HASH_PERFORMANCE 1 in the dhcpd.h file
We are using 6,657 /24 subnets in our pools file
We give out millions IPs each day
Please see below for config files.
This service has been running fine for the last 346 days, last Saturday, for an unknown reason dhcp-1 server had issues, so we turned if off, and dhcp-2 server took over dhcp-1’s part just fine.
So using omapi I told dhcp-2 that its partner was down in order to keep dhp-2 working on its own, so the last log file entry in syslog on dhcp-2 was;
In recent days dhcp-2 is having some difficulty, after comparing a tcpdump to dhcp logs we see DISCOVER requests coming in, but no offers going back out, but this is sporadic, but enough to make users call in.
The last time we saw this issue happen is when the LEASH_HASH size was too low, we are not sure if this is the same issue , maybe because we are only on one server?
Also the dhcpd.leases files grow too big for the /ramdisk, so we are each 10 mins catting /dev/null into /ramdisk/dhcpd.lease! file to save space.
So currently only dhcp-2 is running.
I want to try tonight to simply stop both dhcpd services on both servers, delete the leases file and “touch” new ones then reboot primary first , then secondary a few mins later.
What do you think?
Here are my configs. & Thanks… J
DHCP-1 dhcpd.conf file
-----------
authoritative;
log-facility local7;
db-time-format local;
option domain-name "dev"; # TODO
# DNS internal
option domain-name-servers xxx.xx.xx.210, xxx.xx.xx.220;
default-lease-time 1200; # 20 minutes to match the default Tim Hortons' session duration
max-lease-time 3600; # 1h
# Include EITHER the primary configuration
include "/usr/local/etc/dhcp/dhcpd_primary.conf";
# OR the secondary configuration
#include "/etc/dhcp/dhcpd_secondary.conf";
# No service for the local networks
subnet xxx.xx.0.0 netmask 255.255.255.0 { }
subnet xxx.xx.128.0 netmask 255.255.255.0 { }
subnet xxx.xx.129.0 netmask 255.255.255.0 { }
# All IP ranges for TDL stores
# This file should be automatically generated using the command:
# ./make_ranges.pl < ranges > dhcpd_pools.conf
include "/usr/local/etc/dhcp/dhcpd_pools.conf";
# Non-standard IP ranges (i.e. big stores)
include "/usr/local/etc/dhcp/dhcpd_special_pools.conf";
pid-file-name "/run/dhcpd.pid";
ddns-update-style none;
omapi-port 7911;
omapi-key omapi_key;
key omapi_key {
algorithm hmac-md5;
secret xxxxxxxxxxxxxxxxy==;
}
DHCP-1 dhcpd_primary.conf
## PRIMARY
failover peer "dhcp-failover" {
primary; # declare this to be the primary server
address xxx.xx.xx.9;
port 647;
peer address xxx.xx.xx.11;
peer port 647;
max-response-delay 30;
max-unacked-updates 10;
load balance max seconds 3;
mclt 1800;
split 128;
}
DHCP-2 dhcp-2.conf
----
authoritative;
log-facility local7;
db-time-format local;
option domain-name "tdl"; # TODO
# DV DNS internal
option domain-name-servers XXX.XX.XX.210, XXX.xx.xx.220;
default-lease-time 1200; # 20 minutes to match the default Tim Hortons' session duration
max-lease-time 3600; # 1h
### The below commneted as we are to be independant server - Rob Jan 28th 2016
# Include EITHER the primary configuration
#include "/etc/dhcp/dhcpd_primary.conf";
# OR the secondary configuration
include "/usr/local/etc/dhcpd_secondary.conf";
# No service for the local networks
subnet xxx.xx.0.0 netmask 255.255.255.0 { }
subnet xxx.xx.128.0 netmask 255.255.255.0 { }
subnet xxx.xx.129.0 netmask 255.255.255.0 { }
# All IP ranges for TDL stores
# This file should be automatically generated using the command:
# ./make_ranges.pl < ranges > dhcpd_pools.conf
include "/usr/local/etc/dhcpd_pools.conf";
# Non-standard IP ranges (i.e. big stores)
include "/etc/dhcp/dhcpd_special_pools.conf";
pid-file-name "/run/dhcp-server/dhcpd.pid";
ddns-update-style none;
omapi-port 7911;
omapi-key omapi_key;
key omapi_key {
algorithm hmac-md5;
secret xxxxxxxxxxxxxxx==;
}
DHCP-2 dhcpd_secondary.conf
---
## SECONDARY
failover peer "dhcp-failover" {
secondary;
address XXX.xx.128.11;
port 647;
peer address xxx.xx.128.9;
peer port 647;
max-response-delay 30;
max-unacked-updates 10;
load balance max seconds 3;
}
Dhcp pools file snip it… over 6000 subnets
--
subnet 10.32.0.0 netmask 255.255.255.0 {
option routers 10.32.0.1;
pool {
failover peer "dhcp-failover";
range 10.32.0.5 10.32.0.254;
}
}
…… too long to list J
subnet 10.57.255.0 netmask 255.255.255.0 {
option routers 10.57.255.1;
pool {
failover peer "dhcp-failover";
range 10.57.255.5 10.57.255.254;
}
}
Rob Morin
Gestionnaire des systèmes | Senior Systems Administrator
Tel: 514 385-4448 #174
DATAVALET.COM
5275, chemin Queen-Mary, Montréal (Québec) H3W 1Y3 Canada
CE COURRIEL AINSI QUE CES DOCUMENTS JOINTS peuvent contenir des renseignements confidentiels et privilégiés. Si vous n’êtes pas le destinataire désigné, veuillez nous en informer immédiatement et effacer toute copie. Merci.
THIS EMAIL AND THE DOCUMENTS ATTACHED may contain privileged or confidential information. If the reader of this message is not the intended recipient, please notify the sender immediately and delete the original message. Thank you.
Locked
smime.p7s (6K)