ISC DHCP Users

OMAPI Reservations and peer/failover

Classic

List

Threaded

9 messages Options

Gregory Sloop

OMAPI Reservations and peer/failover

OMAPI Reservations and peer/failover I finally got around to attempting to set the reservation flag on some dhcp leases using omapi.
(I'd asked about how to do this with OMAPI ages ago ... since I can do it live and not have to stop the DHCP servers and hand edit the leases file.)

I'm running a pair of peers/failover/load-balance machines for dhcp.

So, I set the reserved flag on one server, but the leases file on the other server doesn't see/reflect the reservation for that same lease.
(I see the lease on both machines, but there's no reservation on the second server. The one I didn't target with the omapi connection/change.)

Do I have to run OMAPI against both servers?
(I guess I kind of thought since it's a change to the lease, it would get communicated between the peers. That looks like a bad assumption.)

Can someone, (probably ISC) confirm the "right" way to do this?

TIA
-Greg
_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

Gregory Sloop

Re: OMAPI Reservations and peer/failover

Re: OMAPI Reservations and peer/failover Anyone?
I kinda want to get this handled and off my plate, so I'm probably a little impatient. :)

Thanks again!

I finally got around to attempting to set the reservation flag on some dhcp leases using omapi.
(I'd asked about how to do this with OMAPI ages ago ... since I can do it live and not have to stop the DHCP servers and hand edit the leases file.)

I'm running a pair of peers/failover/load-balance machines for dhcp.

So, I set the reserved flag on one server, but the leases file on the other server doesn't see/reflect the reservation for that same lease.
(I see the lease on both machines, but there's no reservation on the second server. The one I didn't target with the omapi connection/change.)

Do I have to run OMAPI against both servers?
(I guess I kind of thought since it's a change to the lease, it would get communicated between the peers. That looks like a bad assumption.)

Can someone, (probably ISC) confirm the "right" way to do this?

TIA
-Greg

_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

Gregory Sloop

Re: OMAPI Reservations and peer/failover

Re: OMAPI Reservations and peer/failover Following up on my own post.
Given what I've been able to dig up on the subject of omapi and peers, I'm pretty sure you have to run against both, explicitly.

But, additional complication arise!

As noted, a fair bit of reading and searching seems to indicate you have to run the omshell commands against each server.
However, this is particularly interesting (or perhaps troubling.)
See: https://lists.isc.org/mailman/htdig/dhcp-users/2006-July/001102.html

To save you the click, I'll quote...
---
"You will have to rerun the statement on both peers.
Take careful note of servers that lose their dhcpd.leases files, you'll have to be able to 0-to-60 them by replaying everything. "
---

There was no expansion on this - and my understanding of it is somewhat ambiguous.
Does this mean that if I have a peer that gets rebuild and the leases file is deleted, it won't get a copy of the "original" leases file from it's peer and that all the "reservation" flags will be lost and I will have to re-run all the omapi commands against the peer which lost the leases file?

Assuming that's the correct interpretation...
I suppose that it's best then, to copy the leases file from the "still-up' peer to the rebuilt peer. (I can't see a reason not to do this, but perhaps I'm missing something.)

I'd be thrilled for the grizzled old-timers out there to weigh in and offer me some of that sage advice! :)
[On this list, anyway, that usually means super-helpful and awesome posts. That isn't the usual response of the old veterans from other lists, however!]

Perhaps this is all done way better in Kea, I dunno. Perhaps I ought to start looking at that.

Thanks in advance!
-Greg

Anyone?
I kinda want to get this handled and off my plate, so I'm probably a little impatient. :)

Thanks again!

Simon Hobson

Re: OMAPI Reservations and peer/failover

Gregory Sloop <[hidden email]> wrote:

> Given what I've been able to dig up on the subject of omapi and peers, I'm pretty sure you have to run against both, explicitly.
>
> But, additional complication arise!
>
> As noted, a fair bit of reading and searching seems to indicate you have to run the omshell commands against each server.
> However, this is particularly interesting (or perhaps troubling.)
> See: https://lists.isc.org/mailman/htdig/dhcp-users/2006-July/001102.html
>
> To save you the click, I'll quote...
> ---
> "You will have to rerun the statement on both peers.
> Take careful note of servers that lose their dhcpd.leases files, you'll have to be able to 0-to-60 them by replaying everything. "
> ---
>
> There was no expansion on this - and my understanding of it is somewhat ambiguous.
> Does this mean that if I have a peer that gets rebuild and the leases file is deleted, it won't get a copy of the "original" leases file from it's peer and that all the "reservation" flags will be lost and I will have to re-run all the omapi commands against the peer which lost the leases file?
>
> Assuming that's the correct interpretation...
> I suppose that it's best then, to copy the leases file from the "still-up' peer to the rebuilt peer. (I can't see a reason not to do this, but perhaps I'm missing something.)

That is indeed an odd statement.
I can completely understand it for a standalone peer - if it loses it's leases file then all your OMAPI made changes will be gone.

But since the a major point of failover is for a server to be able to rebuild itself after a disaster, I would have expected failover to take care of that. I suggest you do a trial with a test server - my expectation is that if you bring up the peer with no leases file, then it'll get everything from it's partner.

However, the warning still holds. Should your shared leases get corrupted or lost, then you'd lose all your changes and need to re-play them. I'm not sure under what circumstances both servers could lose/corrupt their leases, but I'm sure there will be some.
I know that on a single server, if you remove some addresses from the defined ranges, then any leases defined for them will be removed on server startup. So, especially if you build a config and run it out to both servers at once, it's possible to lose a bunch of leases that way.

Simon

_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

Bob Harold

Re: OMAPI Reservations and peer/failover

On Fri, May 28, 2021 at 3:00 PM Simon Hobson <[hidden email]> wrote:

Gregory Sloop <[hidden email]> wrote:

> Given what I've been able to dig up on the subject of omapi and peers, I'm pretty sure you have to run against both, explicitly.
>
> But, additional complication arise!
>
> As noted, a fair bit of reading and searching seems to indicate you have to run the omshell commands against each server.
> However, this is particularly interesting (or perhaps troubling.)
> See: https://lists.isc.org/mailman/htdig/dhcp-users/2006-July/001102.html
>
> To save you the click, I'll quote...
> ---
> "You will have to rerun the statement on both peers.
> Take careful note of servers that lose their dhcpd.leases files, you'll have to be able to 0-to-60 them by replaying everything. "
> ---
>
> There was no expansion on this - and my understanding of it is somewhat ambiguous.
> Does this mean that if I have a peer that gets rebuild and the leases file is deleted, it won't get a copy of the "original" leases file from it's peer and that all the "reservation" flags will be lost and I will have to re-run all the omapi commands against the peer which lost the leases file?
>
> Assuming that's the correct interpretation...
> I suppose that it's best then, to copy the leases file from the "still-up' peer to the rebuilt peer. (I can't see a reason not to do this, but perhaps I'm missing something.)

That is indeed an odd statement.
I can completely understand it for a standalone peer - if it loses it's leases file then all your OMAPI made changes will be gone.

But since the a major point of failover is for a server to be able to rebuild itself after a disaster, I would have expected failover to take care of that. I suggest you do a trial with a test server - my expectation is that if you bring up the peer with no leases file, then it'll get everything from it's partner.

However, the warning still holds. Should your shared leases get corrupted or lost, then you'd lose all your changes and need to re-play them. I'm not sure under what circumstances both servers could lose/corrupt their leases, but I'm sure there will be some.
I know that on a single server, if you remove some addresses from the defined ranges, then any leases defined for them will be removed on server startup. So, especially if you build a config and run it out to both servers at once, it's possible to lose a bunch of leases that way.

Simon

If I understand what Gregory is asking, he is configuring 'host' entries using OMAPI. The failover protocol will copy the 'lease' information, but the 'host' entries would need to be set using OMAPI on each server, and set again if the server loses them. Does that make sense?

Bob Harold

Simon Hobson-2

Re: OMAPI Reservations and peer/failover

Bob Harold <[hidden email]> wrote:

> If I understand what Gregory is asking, he is configuring 'host' entries using OMAPI. The failover protocol will copy the 'lease' information, but the 'host' entries would need to be set using OMAPI on each server, and set again if the server loses them. Does that make sense?

Ah, I'd missed that detail.
I wonder if reserved leases would be more suitable then ?

Simon

_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

Gregory Sloop

Re: OMAPI Reservations and peer/failover

In reply to this post by Bob Harold

Re: OMAPI Reservations and peer/failover Thanks you both for responding.

I'm setting a *reservation* flag on a lease in the leases file.
The list posting I reference is to a new lease def to the leases file.

Setting up a test, while a fair bit of work, might be interesting, if nothing more definitive comes along.
I guess I could simply take one of the peers down, delete/rename the lease file, and bring it back up and see how it's populated. (That would be easy, if somewhat more risky. LOL)

Would there be any complications on copying the leases file from the still-standing DHCP pair that you can think of?
(I _think_ they should be identical, provided there's not been lost communication between the peers and they are out of sync for that reason, and there's lost information in the down/dead pair. But that information is going to be lost in any case (provided the dead peer truly can't be raised from the dead), so any ripples from that loss won't be avoidable in any case.)

These servers, at least in the case here are in VM's which are pretty easily and quickly restored from backups and such, so I'm considering third and fourth order disasters - but sometimes things happen. I'd like to be ready when/if they do. :)

Again, thanks for your thoughts!

-Greg

On Fri, May 28, 2021 at 3:00 PM Simon Hobson <[hidden email]> wrote:

Gregory Sloop <[hidden email]> wrote:

> Given what I've been able to dig up on the subject of omapi and peers, I'm pretty sure you have to run against both, explicitly.
>
> But, additional complication arise!
>
> As noted, a fair bit of reading and searching seems to indicate you have to run the omshell commands against each server.
> However, this is particularly interesting (or perhaps troubling.)
> See: https://lists.isc.org/mailman/htdig/dhcp-users/2006-July/001102.html
>
> To save you the click, I'll quote...
> ---
> "You will have to rerun the statement on both peers.
> Take careful note of servers that lose their dhcpd.leases files, you'll have to be able to 0-to-60 them by replaying everything. "
> ---
>
> There was no expansion on this - and my understanding of it is somewhat ambiguous.
> Does this mean that if I have a peer that gets rebuild and the leases file is deleted, it won't get a copy of the "original" leases file from it's peer and that all the "reservation" flags will be lost and I will have to re-run all the omapi commands against the peer which lost the leases file?
>
> Assuming that's the correct interpretation...
> I suppose that it's best then, to copy the leases file from the "still-up' peer to the rebuilt peer. (I can't see a reason not to do this, but perhaps I'm missing something.)

That is indeed an odd statement.
I can completely understand it for a standalone peer - if it loses it's leases file then all your OMAPI made changes will be gone.

But since the a major point of failover is for a server to be able to rebuild itself after a disaster, I would have expected failover to take care of that. I suggest you do a trial with a test server - my expectation is that if you bring up the peer with no leases file, then it'll get everything from it's partner.

However, the warning still holds. Should your shared leases get corrupted or lost, then you'd lose all your changes and need to re-play them. I'm not sure under what circumstances both servers could lose/corrupt their leases, but I'm sure there will be some.
I know that on a single server, if you remove some addresses from the defined ranges, then any leases defined for them will be removed on server startup. So, especially if you build a config and run it out to both servers at once, it's possible to lose a bunch of leases that way.

Simon

glenn.satchell

Re: OMAPI Reservations and peer/failover

In reply to this post by Simon Hobson

On 2021-05-29 05:00, Simon Hobson wrote:

> Gregory Sloop <[hidden email]> wrote:
>
>> Given what I've been able to dig up on the subject of omapi and peers,
>> I'm pretty sure you have to run against both, explicitly.
>>
>> But, additional complication arise!
>>
>> As noted, a fair bit of reading and searching seems to indicate you
>> have to run the omshell commands against each server.
>> However, this is particularly interesting (or perhaps troubling.)
>> See:
>> https://lists.isc.org/mailman/htdig/dhcp-users/2006-July/001102.html
>>
>> To save you the click, I'll quote...
>> ---
>> "You will have to rerun the statement on both peers.
>> Take careful note of servers that lose their dhcpd.leases files,
>> you'll have to be able to 0-to-60 them by replaying everything. "
>> ---
>>
>> There was no expansion on this - and my understanding of it is
>> somewhat ambiguous.
>> Does this mean that if I have a peer that gets rebuild and the leases
>> file is deleted, it won't get a copy of the "original" leases file
>> from it's peer and that all the "reservation" flags will be lost and I
>> will have to re-run all the omapi commands against the peer which lost
>> the leases file?
>>
>> Assuming that's the correct interpretation...
>> I suppose that it's best then, to copy the leases file from the
>> "still-up' peer to the rebuilt peer. (I can't see a reason not to do
>> this, but perhaps I'm missing something.)
>
> That is indeed an odd statement.
> I can completely understand it for a standalone peer - if it loses
> it's leases file then all your OMAPI made changes will be gone.
>
> But since the a major point of failover is for a server to be able to
> rebuild itself after a disaster, I would have expected failover to
> take care of that. I suggest you do a trial with a test server - my
> expectation is that if you bring up the peer with no leases file, then
> it'll get everything from it's partner.
>
> However, the warning still holds. Should your shared leases get
> corrupted or lost, then you'd lose all your changes and need to
> re-play them. I'm not sure under what circumstances both servers could
> lose/corrupt their leases, but I'm sure there will be some.
> I know that on a single server, if you remove some addresses from the
> defined ranges, then any leases defined for them will be removed on
> server startup. So, especially if you build a config and run it out to
> both servers at once, it's possible to lose a bunch of leases that
> way.
>
> Simon

Instead of using a reserved lease, how about just setting it to a really
long time instead, say 3, 6 or 12 months? Use a class or group for these
special leases. Since that goes in dhcpd.conf it will survive loss of
the leases file.

Also note that while the leases recorded in the leases file are the same
for a failover peer, the actual files are not identical. Various
settings such as "failover peer state", tstp, tsfp, atsfp, cltt can be
different. "binding state backup" is a special state for failover. So
you can't just copy one dhcpd.leases file to the other server.

With a failover setup, the two servers should synchronise their leases
by themselves, thus a server with an empty leases file should build up
the information from the other server. You would probably need to look
at the source to see what fields get copied over during this process, eg
the reserved field.

So in server/db.c write_lease() has a statement to write out the
reserved statement to dhcpd.leases:
if (lease->flags & RESERVED_LEASE)
if (fprintf(db_file, "\n reserved;") < 0)
++errors;

but I still don't know if this gets replicated in a failover
configuration. Probably need to look in server/failover.c at
dhcp_failover_startup()

regards,
Glenn
_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

Gregory Sloop

Re: OMAPI Reservations and peer/failover

Re: OMAPI Reservations and peer/failover Top posting:

gsuca> the information from the other server. You would probably need to look
gsuca> at the source to see what fields get copied over during this process, eg
gsuca> the reserved field

Source code?! You would probably suggest that I run with scissors too, eh!?
(Me reading source code could be at least as dangerous!)
:)

So, I just just took the easy route; Stopped the peer (non-master/primary peer - though I think it should be fine either way), deleted the leases file, started it back up, and waited (MCLT/2) time, till they came back to normal.
All the reserved flags ARE set in the peers' leases file, where I deleted the leases file.

So, I think that's a definitive answer.
So, at least for reserved flags (I can't answer for anything else) once set on both peers, if one peer is lost and rebuilt from scratch, the reserved flags will get populated from the still surviving peer in the leases file..

Again, thanks all for the input.

-Greg

gsuca> On 2021-05-29 05:00, Simon Hobson wrote:
>> Gregory Sloop <[hidden email]> wrote:

>>> Given what I've been able to dig up on the subject of omapi and peers,
>>> I'm pretty sure you have to run against both, explicitly.

>>> But, additional complication arise!

>>> As noted, a fair bit of reading and searching seems to indicate you
>>> have to run the omshell commands against each server.
>>> However, this is particularly interesting (or perhaps troubling.)
>>> See:
>>> https://lists.isc.org/mailman/htdig/dhcp-users/2006-July/001102.html

>>> To save you the click, I'll quote...
>>> ---
>>> "You will have to rerun the statement on both peers.
>>> Take careful note of servers that lose their dhcpd.leases files,
>>> you'll have to be able to 0-to-60 them by replaying everything. "
>>> ---

>>> There was no expansion on this - and my understanding of it is
>>> somewhat ambiguous.
>>> Does this mean that if I have a peer that gets rebuild and the leases
>>> file is deleted, it won't get a copy of the "original" leases file
>>> from it's peer and that all the "reservation" flags will be lost and I
>>> will have to re-run all the omapi commands against the peer which lost
>>> the leases file?

>>> Assuming that's the correct interpretation...
>>> I suppose that it's best then, to copy the leases file from the
>>> "still-up' peer to the rebuilt peer. (I can't see a reason not to do
>>> this, but perhaps I'm missing something.)

>> That is indeed an odd statement.
>> I can completely understand it for a standalone peer - if it loses
>> it's leases file then all your OMAPI made changes will be gone.

>> But since the a major point of failover is for a server to be able to
>> rebuild itself after a disaster, I would have expected failover to
>> take care of that. I suggest you do a trial with a test server - my
>> expectation is that if you bring up the peer with no leases file, then
>> it'll get everything from it's partner.

>> However, the warning still holds. Should your shared leases get
>> corrupted or lost, then you'd lose all your changes and need to
>> re-play them. I'm not sure under what circumstances both servers could
>> lose/corrupt their leases, but I'm sure there will be some.
>> I know that on a single server, if you remove some addresses from the
>> defined ranges, then any leases defined for them will be removed on
>> server startup. So, especially if you build a config and run it out to
>> both servers at once, it's possible to lose a bunch of leases that
>> way.

>> Simon

gsuca> Instead of using a reserved lease, how about just setting it to a really
gsuca> long time instead, say 3, 6 or 12 months? Use a class or group for these
gsuca> special leases. Since that goes in dhcpd.conf it will survive loss of
gsuca> the leases file.

gsuca> Also note that while the leases recorded in the leases file are the same
gsuca> for a failover peer, the actual files are not identical. Various
gsuca> settings such as "failover peer state", tstp, tsfp, atsfp, cltt can be
gsuca> different. "binding state backup" is a special state for failover. So
gsuca> you can't just copy one dhcpd.leases file to the other server.

gsuca> With a failover setup, the two servers should synchronise their leases
gsuca> by themselves, thus a server with an empty leases file should build up
gsuca> the information from the other server. You would probably need to look
gsuca> at the source to see what fields get copied over during this process, eg
gsuca> the reserved field.

gsuca> So in server/db.c write_lease() has a statement to write out the
gsuca> reserved statement to dhcpd.leases:
gsuca> if (lease->flags & RESERVED_LEASE)
gsuca> if (fprintf(db_file, "\n reserved;") < 0)
gsuca> ++errors;

gsuca> but I still don't know if this gets replicated in a failover
gsuca> configuration. Probably need to look in server/failover.c at
gsuca> dhcp_failover_startup()

gsuca> regards,
gsuca> Glenn
gsuca> _______________________________________________
gsuca> ISC funds the development of this software with paid support
gsuca> subscriptions. Contact us at https://www.isc.org/contact/ for more information.

gsuca> dhcp-users mailing list
[hidden email]
gsuca> https://lists.isc.org/mailman/listinfo/dhcp-users

_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users