Hostname differences between servers

[Norman Elton]

In our environment, we have multiple routers (VRRP) forwarding
requests to two DHCP servers, run in failover. So if a client
broadcasts out a DHCPREQUEST, that message is handled multiple times.
Not a big deal, it's been working great for years.

We've just noticed that, sometimes, the hostname reported by the two
servers don't match each other. For instance:

Oct  1 01:03:48 landlord01: DHCPREQUEST for 10.45.13.196
(128.239.10.121) from e8802ea2eb7f (Michaels-iPhone) via 10.45.13.2
Oct  1 01:03:48 landlord02: DHCPREQUEST for 10.45.13.196
(128.239.10.121) from e8802ea2eb7f (android-9375f90cad51e32f) via
10.45.13.2: lease owned by peer

So, is it an android, or is it an iPhone? In this case, the mac
address points to Apple, but why is one server reporting the hostname
is android-9375f90cad51e32f?

This is happening fairly frequently. Our servers are running the
RHEL-provided version of ISC dhcpd, 4.1.1-61.P1.el6_10.

Any ideas?

Thanks,

Norman
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

[Eugene Grosbein]

02.10.2018 21:53, Norman Elton wrote:

> In our environment, we have multiple routers (VRRP) forwarding
> requests to two DHCP servers, run in failover. So if a client
> broadcasts out a DHCPREQUEST, that message is handled multiple times.
> Not a big deal, it's been working great for years.
>
> We've just noticed that, sometimes, the hostname reported by the two
> servers don't match each other. For instance:
>
> Oct  1 01:03:48 landlord01: DHCPREQUEST for 10.45.13.196
> (128.239.10.121) from e8802ea2eb7f (Michaels-iPhone) via 10.45.13.2
> Oct  1 01:03:48 landlord02: DHCPREQUEST for 10.45.13.196
> (128.239.10.121) from e8802ea2eb7f (android-9375f90cad51e32f) via
> 10.45.13.2: lease owned by peer
>
> So, is it an android, or is it an iPhone? In this case, the mac
> address points to Apple, but why is one server reporting the hostname
> is android-9375f90cad51e32f?
>
> This is happening fairly frequently. Our servers are running the
> RHEL-provided version of ISC dhcpd, 4.1.1-61.P1.el6_10.
>
> Any ideas?

First, you should use tcpdump -enpvvvs0 to see more details of requests
and look for differences as they have much more parts than dhcp servers shows.

You may be having un- or mis-configured dhcp relay/proxy in the network duplicating requests.

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

[Norman Elton]

An update after further research!

In the previous example, e8802ea2eb7f (an Apple device) is requesting
10.45.13.196. One server, landlord02, is erroneously stating that the
client hostname is android-9375f90cad51e32f. It turns out that, a few
days ago, landlord02 offered this IP address to an Android device with
the hostname of android-9375f90cad51e32f. Looking at the source code,
it appears that dhcpd is not reporting the hostname found in the
packet, but rather, the information collected from its lease file.

I suspect this is a corner case, related to the failover mechanism.
But is there a reason it's logging from the lease file and not from
the packet itself? The log message could reasonably be interpreted to
mean that e8802ea2eb7f is reporting an Android hostname.

Thanks!

Norman
On Tue, Oct 2, 2018 at 10:53 AM Norman Elton <[hidden email]> wrote:

>
> In our environment, we have multiple routers (VRRP) forwarding
> requests to two DHCP servers, run in failover. So if a client
> broadcasts out a DHCPREQUEST, that message is handled multiple times.
> Not a big deal, it's been working great for years.
>
> We've just noticed that, sometimes, the hostname reported by the two
> servers don't match each other. For instance:
>
> Oct  1 01:03:48 landlord01: DHCPREQUEST for 10.45.13.196
> (128.239.10.121) from e8802ea2eb7f (Michaels-iPhone) via 10.45.13.2
> Oct  1 01:03:48 landlord02: DHCPREQUEST for 10.45.13.196
> (128.239.10.121) from e8802ea2eb7f (android-9375f90cad51e32f) via
> 10.45.13.2: lease owned by peer
>
> So, is it an android, or is it an iPhone? In this case, the mac
> address points to Apple, but why is one server reporting the hostname
> is android-9375f90cad51e32f?
>
> This is happening fairly frequently. Our servers are running the
> RHEL-provided version of ISC dhcpd, 4.1.1-61.P1.el6_10.
>
> Any ideas?
>
> Thanks,
>
> Norman

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

[Norman Elton]

Eugene -

Thanks for the tip, I didn't see your response until just now. I did
do a packet capture, and caught one of these "in the wild". The
packets coming into dhcpd are consistently using one hostname, but the
two servers are logging two different hostnames. In the previous
example, for instance, the packets would show Michaels-iPhone, but one
of the servers is erroneously logging android-9375f90cad51e32f.

Thanks!

Norman
On Tue, Oct 2, 2018 at 1:59 PM Eugene Grosbein <[hidden email]> wrote:

>
> 02.10.2018 21:53, Norman Elton wrote:
>
> > In our environment, we have multiple routers (VRRP) forwarding
> > requests to two DHCP servers, run in failover. So if a client
> > broadcasts out a DHCPREQUEST, that message is handled multiple times.
> > Not a big deal, it's been working great for years.
> >
> > We've just noticed that, sometimes, the hostname reported by the two
> > servers don't match each other. For instance:
> >
> > Oct  1 01:03:48 landlord01: DHCPREQUEST for 10.45.13.196
> > (128.239.10.121) from e8802ea2eb7f (Michaels-iPhone) via 10.45.13.2
> > Oct  1 01:03:48 landlord02: DHCPREQUEST for 10.45.13.196
> > (128.239.10.121) from e8802ea2eb7f (android-9375f90cad51e32f) via
> > 10.45.13.2: lease owned by peer
> >
> > So, is it an android, or is it an iPhone? In this case, the mac
> > address points to Apple, but why is one server reporting the hostname
> > is android-9375f90cad51e32f?
> >
> > This is happening fairly frequently. Our servers are running the
> > RHEL-provided version of ISC dhcpd, 4.1.1-61.P1.el6_10.
> >
> > Any ideas?
>
> First, you should use tcpdump -enpvvvs0 to see more details of requests
> and look for differences as they have much more parts than dhcp servers shows.
>
> You may be having un- or mis-configured dhcp relay/proxy in the network duplicating requests.
>

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

[Eugene Grosbein]

03.10.2018 2:13, Norman Elton wrote:

> An update after further research!
>
> In the previous example, e8802ea2eb7f (an Apple device) is requesting
> 10.45.13.196. One server, landlord02, is erroneously stating that the
> client hostname is android-9375f90cad51e32f. It turns out that, a few
> days ago, landlord02 offered this IP address to an Android device with
> the hostname of android-9375f90cad51e32f. Looking at the source code,
> it appears that dhcpd is not reporting the hostname found in the
> packet, but rather, the information collected from its lease file.
>
> I suspect this is a corner case, related to the failover mechanism.
> But is there a reason it's logging from the lease file and not from
> the packet itself? The log message could reasonably be interpreted to
> mean that e8802ea2eb7f is reporting an Android hostname.

And it did report that hostname. You should not have different hosts having same
MAC address in the network, and dhcpd cannot differentiate them anyway,
so it uses same lease.

If you have DHCP servers separated from users vlans by routers and DHCP relays
and your users differ not only by MAC but may have same MACs within differed vlans
identified by option 82, for example, than you need DHCP server other than ISC DHCPD.
FreeRADIUS with its DHCP module serves me just fine in such setup.

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

[Neufeld, Keith]

In the previous example, e8802ea2eb7f (an Apple device) is requesting
10.45.13.196. One server, landlord02, is erroneously stating that the
client hostname is android-9375f90cad51e32f. It turns out that, a few
days ago, landlord02 offered this IP address to an Android device with
the hostname of android-9375f90cad51e32f. Looking at the source code,
it appears that dhcpd is not reporting the hostname found in the
packet, but rather, the information collected from its lease file.

Norman, since we see the REQUEST relayed by your router, it’s a broadcast REQUEST.  I’m assuming this is part of the initial broadcast DISCOVER - OFFER - REQUEST - ACK sequence, and not a later REQUEST being broadcast?  (That can happen but only under special circumstances.)

I speculate that this happened:

1.  The client did a broadcast DISCOVER that was relayed to both servers.  Possibly both of them logged the correct client-hostname in their DISCOVER messages.

2.  Both servers sent OFFERs back toward the client via the relay.  landlord01’s OFFER was for 10.45.13.196 ; landlord02’s OFFER was for some other IP.

3.  The client decided to accept landlord01’s offer and broadcast a REQUEST for that offer.

4.  landlord01 received the REQUEST and logged it correctly.

5.  landlord02 received the REQUEST and, recognizing that it was in response to an OFFER made by its peer, logged it without inspecting the packet for current supplemental data, leaving in place the cached client-hostname from the last client to hold this lease (IP) when offered by landlord02.

That is to say, client-hostname tends to stay cached a long time, misleadingly, in the lease database of the peer of the server that’s handling the client currently holding that lease / IP.

That’s … more difficult to phrase than I expected it to be.  Perhaps there’s an easier way to describe it.

-- 

Keith Neufeld

Director of Networking, Telecommunications, and IT Security

Wichita State University

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

[Norman Elton]

Keith gets a gold star for explaining it more clearly. My head is
hurting from staring at log files all day! Rather than interpret the
log files, perhaps it's easiest just to share the relevant snippets:

## ANDROID DEVICE COMING ONLINE
Sep 30 13:53:46 landlord01: DHCPDISCOVER from c0174ddd00a4
(android-9375f90cad51e32f) via 10.45.13.2
Sep 30 13:53:46 landlord02: DHCPDISCOVER from c0174ddd00a4 via 10.45.13.2
Sep 30 13:53:46 landlord01: DHCPOFFER on 10.45.13.73 to c0174ddd00a4
(android-9375f90cad51e32f) via 10.45.13.2
Sep 30 13:53:46 landlord02: uid lease 10.45.13.196 for client
c0174ddd00a4 is duplicate on INIT-MONROE
Sep 30 13:53:46 landlord02: DHCPOFFER on 10.45.13.196 to c0174ddd00a4
(android-9375f90cad51e32f) via 10.45.13.2
Sep 30 13:53:46 landlord02: DHCPREQUEST for 10.45.13.73
(128.239.10.121) from c0174ddd00a4 via 10.45.13.2: lease owned by peer
Sep 30 13:53:46 landlord01: DHCPREQUEST for 10.45.13.73
(128.239.10.121) from c0174ddd00a4 (android-9375f90cad51e32f) via
10.45.13.2
Sep 30 13:53:46 landlord01: DHCPACK on 10.45.13.73 to c0174ddd00a4
(android-9375f90cad51e32f) via 10.45.13.2

## IPHONE COMING ONLINE
Oct  1 01:03:46 landlord01: DHCPDISCOVER from e8802ea2eb7f via 10.45.13.2
Oct  1 01:03:46 landlord02: DHCPDISCOVER from e8802ea2eb7f via
10.45.13.2: load balance to peer wm-dhcp-01-02
Oct  1 01:03:46 landlord01: DHCPOFFER on 10.45.13.196 to e8802ea2eb7f
(Michaels-iPhone) via 10.45.13.2
Oct  1 01:03:48 landlord02: DHCPREQUEST for 10.45.13.196
(128.239.10.121) from e8802ea2eb7f (android-9375f90cad51e32f) via
10.45.13.2: lease owned by peer
Oct  1 01:03:48 landlord01: DHCPREQUEST for 10.45.13.196
(128.239.10.121) from e8802ea2eb7f (Michaels-iPhone) via 10.45.13.2
Oct  1 01:03:48 landlord01: DHCPACK on 10.45.13.196 to e8802ea2eb7f
(Michaels-iPhone) via 10.45.13.2

Landlord02 offers 10.45.13.196 to the Android on September 30th, then
regurgitates that hostname on October 1st when the iPhone is
connecting.

Question is ... why does dhcpd log the hostname from its lease file
when that lease is irrelevant to the client in question?

Norman

On Tue, Oct 2, 2018 at 3:50 PM Neufeld, Keith <[hidden email]> wrote:

>
> In the previous example, e8802ea2eb7f (an Apple device) is requesting
> 10.45.13.196. One server, landlord02, is erroneously stating that the
> client hostname is android-9375f90cad51e32f. It turns out that, a few
> days ago, landlord02 offered this IP address to an Android device with
> the hostname of android-9375f90cad51e32f. Looking at the source code,
> it appears that dhcpd is not reporting the hostname found in the
> packet, but rather, the information collected from its lease file.
>
>
> Norman, since we see the REQUEST relayed by your router, it’s a broadcast REQUEST.  I’m assuming this is part of the initial broadcast DISCOVER - OFFER - REQUEST - ACK sequence, and not a later REQUEST being broadcast?  (That can happen but only under special circumstances.)
>
> I speculate that this happened:
>
> 1.  The client did a broadcast DISCOVER that was relayed to both servers.  Possibly both of them logged the correct client-hostname in their DISCOVER messages.
> 2.  Both servers sent OFFERs back toward the client via the relay.  landlord01’s OFFER was for 10.45.13.196 ; landlord02’s OFFER was for some other IP.
> 3.  The client decided to accept landlord01’s offer and broadcast a REQUEST for that offer.
> 4.  landlord01 received the REQUEST and logged it correctly.
> 5.  landlord02 received the REQUEST and, recognizing that it was in response to an OFFER made by its peer, logged it without inspecting the packet for current supplemental data, leaving in place the cached client-hostname from the last client to hold this lease (IP) when offered by landlord02.
>
> That is to say, client-hostname tends to stay cached a long time, misleadingly, in the lease database of the peer of the server that’s handling the client currently holding that lease / IP.
>
> That’s … more difficult to phrase than I expected it to be.  Perhaps there’s an easier way to describe it.
>
> --
> Keith Neufeld
> Director of Networking, Telecommunications, and IT Security
> Wichita State University
>
> _______________________________________________
> dhcp-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users