DHCP lease logging between peers

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

DHCP lease logging between peers

Nate Collins
Hello,

I'm looking to inquire about the logging behavior of isc-dhcp-server when running with a peer, and whether or not our environment is configured correctly to allow the DHCP transaction logs to be shared between the two.

Our network has two servers running isc-dhcpd-4.3.3 configured as failover peers (configuration below). Both of these servers are logging to /var/log/syslog, and both servers log the transactions of the other server:

server1:

 # cat /var/log/syslog | grep -i b8:27:eb:00:11:22
Jul 15 08:51:31 server1 dhcpd[1303]: DHCPREQUEST for 10.0.7.143 from b8:27:eb:00:11:22 via bond0
Jul 15 08:51:31 server1 dhcpd[1303]: DHCPACK on 10.0.7.143 to b8:27:eb:00:11:22 via bond0

server2:

 # cat /var/log/syslog | grep -i b8:27:eb:00:11:22
Jul 15 08:51:31 server2 dhcpd[19642]: DHCPREQUEST for 10.0.7.143 from b8:27:eb:00:11:22 via bond0
Jul 15 08:51:31 server2 dhcpd[19642]: DHCPACK on 10.0.7.143 to b8:27:eb:00:11:22 via bond0

Occasionally, however, we will see it where a static DHCP lease is handed out by one server, but the lease is not logged in the syslog file of the other server, despite being in both dhcpd.leases files:

server1:

 # cat /var/log/syslog | grep -i b8:27:eb:ab:cd:ef
...
Jul 12 07:33:03 server1 dhcpd[2094]: DHCPREQUEST for 10.0.7.243 from b8:27:eb:ab:cd:ef via bond0
Jul 12 07:33:03 server1 dhcpd[2094]: DHCPACK on 10.0.7.243 to b8:27:eb:ab:cd:ef via bond0

server2:

 # cat /var/log/syslog | grep -i b8:27:eb:ab:cd:ef
 #

Note that the logs haven't been rotated. We notice this going both ways (primary -> secondary, secondary -> primary), for both static leases and for the dynamic address pool.

The failover configuration of the two servers are as follows (let me know if more is needed to debug this issue):

# server1 /etc/dhcp/dhcpd.conf:

failover peer dhcp-failover {
    primary;
    address 10.0.0.201;
    port 647;
    peer address 10.0.0.202;
    peer port 647;
    max-response-delay 60;
    max-unacked-updates 10;
    load balance max seconds 8;
    mclt 3600;
    split 128;
}

# server2 /etc/dhcp/dhcpd.conf:

failover peer dhcp-failover {
    secondary;
    address 10.0.0.202;
    port 647;
    peer address 10.0.0.201;
    peer port 647;
    max-response-delay 60;
    max-unacked-updates 10;
    load balance max seconds 8;
}

Ultimately we'd like it so that all DHCP transactions from one server is logged by the other server; is there some configuration option corresponding to this? Again, let me know if more sections from our configuration or more information about our layout is needed.

Thanks.
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: DHCP lease logging between peers

Simon Hobson
Nate Collins <[hidden email]> wrote:

> I'm looking to inquire about the logging behavior of isc-dhcp-server when running with a peer, and whether or not our environment is configured correctly to allow the DHCP transaction logs to be shared between the two.
>
> Our network has two servers running isc-dhcpd-4.3.3 configured as failover peers (configuration below). Both of these servers are logging to /var/log/syslog, and both servers log the transactions of the other server:
>
> server1:
>
> # cat /var/log/syslog | grep -i b8:27:eb:00:11:22
> Jul 15 08:51:31 server1 dhcpd[1303]: DHCPREQUEST for 10.0.7.143 from b8:27:eb:00:11:22 via bond0
> Jul 15 08:51:31 server1 dhcpd[1303]: DHCPACK on 10.0.7.143 to b8:27:eb:00:11:22 via bond0
>
> server2:
>
> # cat /var/log/syslog | grep -i b8:27:eb:00:11:22
> Jul 15 08:51:31 server2 dhcpd[19642]: DHCPREQUEST for 10.0.7.143 from b8:27:eb:00:11:22 via bond0
> Jul 15 08:51:31 server2 dhcpd[19642]: DHCPACK on 10.0.7.143 to b8:27:eb:00:11:22 via bond0
>
> Occasionally, however, we will see it where a static DHCP lease is handed out by one server, but the lease is not logged in the syslog file of the other server, despite being in both dhcpd.leases files:
>
> server1:
>
> # cat /var/log/syslog | grep -i b8:27:eb:ab:cd:ef
> ...
> Jul 12 07:33:03 server1 dhcpd[2094]: DHCPREQUEST for 10.0.7.243 from b8:27:eb:ab:cd:ef via bond0
> Jul 12 07:33:03 server1 dhcpd[2094]: DHCPACK on 10.0.7.243 to b8:27:eb:ab:cd:ef via bond0
>
> server2:
>
> # cat /var/log/syslog | grep -i b8:27:eb:ab:cd:ef
> #

You would need to see what the clients are doing - eg packet capture on the network.
I would hazard a guess that in example 1, the client broadcast the request and so it was picked up by both servers - this is likely when the client connects to the network and without further tests (which some clients do) cannot know before doing the DHCP request whether it is still on the same network where it got it's lease from. Since both servers have a copy of the lease, I guess both are replying - I don't know if that's correct behaviour as I'm not familiar enough with failover.
In example 2, I suspect the client has unicast it's request to the server it got it's lease from. This is normal where the client already has a working network connection and just wants to renew it's currently active lease. The other server will never see the unicast packet, so it will neither respond to the client nor log anything.



> Note that the logs haven't been rotated. We notice this going both ways (primary -> secondary, secondary -> primary), for both static leases and for the dynamic address pool.

As above, I think this is normal behaviour.

> Ultimately we'd like it so that all DHCP transactions from one server is logged by the other server

I don't think this is possible since each server will not see requests unicast to the other server.

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: DHCP lease logging between peers

Niall O'Reilly
In reply to this post by Nate Collins

On 15 Jul 2019, at 15:00, Nate Collins wrote:

Occasionally, however, we will see it where a static DHCP lease is handed out by one server, but the lease is not logged in the syslog file of the other server, despite being in both dhcpd.leases files:

If I've understood correctly both

  • what you wish to achieve, and

  • how the software works,

then I think you may be wishing for something which is not available.

Static DHCP leases are defined in host { ... } declarations.
As you probably know, a common method for ensuring resilience is
to provision the same host { ... } declarations in the configuration
of each of a number of servers.

Dynamic leases are assigned from pools defined in pool { ... } declarations.
Failover peer relationships are also defined in pool { ... } declarations.

The host { ... } and pool { ... } declarations don't (or shouldn't)
intersect, as the former are in global scope, and the latter in subnet scope.
Consequently, even identifying the appropriate failover peer to notify when
a static lease is (re-)assigned (let alone delivering the notification) is
not possible.

I hope I've not misunderstood, and that this helps.

Best regards,
Niall O'Reilly


_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: DHCP lease logging between peers

Nate Collins
In reply to this post by Simon Hobson
Simon Hobson wrote:

> You would need to see what the clients are doing - eg packet capture on the network.
> I would hazard a guess that in example 1, the client broadcast the request and so it was picked up by both servers - this is likely when the client connects to the network and without further tests (which some clients do) cannot know before doing the DHCP request whether it is still on the same network where it got it's lease from.
> In example 2, I suspect the client has unicast it's request to the server it got it's lease from. This is normal where the client already has a working network connection and just wants to renew it's currently active lease. The other server will never see the unicast packet, so it will neither respond to the client nor log anything.

> I don't think this is possible since each server will not see requests unicast to the other server.

I see - I had assumed that the duplicate log messages between the two servers (like in example 1) were due to the dhcpd daemons communicating to keep their logs in sync, rather than the DHCP server simply seeing the exchange on the network. I'll do some wiresharking to confirm this, but ultimately if I shouldn't rely on the syslog file for up-to-date information between peers, my main question is answered.

> Since both servers have a copy of the lease, I guess both are replying - I don't know if that's correct behaviour as I'm not familiar enough with failover.

I agree definitely that this is what it looks like, but from what I understand, the `split 128;` should evenly load balance the client between the two servers, and ultimately only one would reply (which is why I assumed the duplicate DHCPACKs were just the servers logging each other's transactions through the daemon or something).

Niall O'Reilly wrote:

> Static DHCP leases are defined in host { ... } declarations.
> As you probably know, a common method for ensuring resilience is
> to provision the same host { ... } declarations in the configuration
> of each of a number of servers.

> Dynamic leases are assigned from pools defined in pool { ... } declarations.
> Failover peer relationships are also defined in pool { ... } declarations.

> The host { ... } and pool { ... } declarations don't (or shouldn't)
> intersect, as the former are in global scope, and the latter in subnet scope.

Right - our `host { ... }` declarations are the same between servers, as well as the pools (which I believe should be evenly load balanced given our configuration), which don't intersect.

> then I think you may be wishing for something which is not available.

> Consequently, even identifying the appropriate failover peer to notify when
> a static lease is (re-)assigned (let alone delivering the notification) is
> not possible.

> I hope I've not misunderstood, and that this helps.

This makes sense (and goes along with the shared transaction logs being due to multicast packets), since I see static leases aren't held in the dhcpd.leases file. Thanks for the input.
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: DHCP lease logging between peers

Simon Hobson
Nate Collins <[hidden email]> wrote:

>> Since both servers have a copy of the lease, I guess both are replying - I don't know if that's correct behaviour as I'm not familiar enough with failover.
>
> I agree definitely that this is what it looks like, but from what I understand, the `split 128;` should evenly load balance the client between the two servers, and ultimately only one would reply ...

I think there's more to it than that.
When a client does a DHCP-Discover, then the split statement does affect how the servers respond. But if the client does a DHCP-Request (broadcast) for a specific address, then both servers are in a position to respond if it matches an existing lease. As I said, I'm not really familiar with failover so I can't really say if both respond "because they both can" or whether only the master for that lease should respond.

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users