dns_update_log shows dynamic entries deleted

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

dns_update_log shows dynamic entries deleted

Cuttler, Brian (HEALTH)

We are running on Centos 7.1 and have dhcp 4.2.5, with bind 9.9.4.

 

We are running dhcp with subnet 10.57.36.0/22 permitted dynamic dns udpates, except for the last 100 addresses in the pool, which we excluded to allow for static addresses that we needed.

 

Things have been working wonderfully – but at 08:18 this morning a large number of entries were removed from dynamic dns.

 

The dhcp clients were either, only asleep for the night (Windows, printers) or actually online (linux).

 

I’ve no idea why these DNS entries were removed, exactly how to get them back, and how to prevent them from being removed again.

 

I’d suspect it was a lease expiration issue, but these machines all check in and renew, at least they should be, I can check the boot.log and see.

 

Thanks for your help,

Brian

 


_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

RE: dns_update_log shows dynamic entries deleted

Cuttler, Brian (HEALTH)

I disabled update_optimization and as dhcp releases are renewed (I already has update_static enabled) we are re-creating the DNS entries.

 

I still have no idea why those entries all got deleted and am rather stressed over it.

 

Please help me to understand what went wrong, and how to prevent or properly fix.

 

Many thanks,

Brian

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Cuttler, Brian (HEALTH)
Sent: Tuesday, June 09, 2015 12:05 PM
To: Users of ISC DHCP
Subject: dns_update_log shows dynamic entries deleted

 

We are running on Centos 7.1 and have dhcp 4.2.5, with bind 9.9.4.

 

We are running dhcp with subnet 10.57.36.0/22 permitted dynamic dns udpates, except for the last 100 addresses in the pool, which we excluded to allow for static addresses that we needed.

 

Things have been working wonderfully – but at 08:18 this morning a large number of entries were removed from dynamic dns.

 

The dhcp clients were either, only asleep for the night (Windows, printers) or actually online (linux).

 

I’ve no idea why these DNS entries were removed, exactly how to get them back, and how to prevent them from being removed again.

 

I’d suspect it was a lease expiration issue, but these machines all check in and renew, at least they should be, I can check the boot.log and see.

 

Thanks for your help,

Brian

 


_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: dns_update_log shows dynamic entries deleted

dave c
Well, you are likely asking the wrong side of the equation.

If I had to venture a guess, your DNS server reloaded itself.

Dynamic DNS updates are written to a local cache for the zone. I don't know if that cache will
always persist through having DNS restarted, but I'd suggest that as it's dynamic, it's not
designed to be persistent in the DNS zone. Otherwise it would be in a DNS zone file :)

Do you have any info or stats on the DNS side of things?

If you had said that you started seeing updates not getting through to DNS, then I'd suggest
that the issue might be that the DHCP system and DNS system had a change in what they used to
authenticate updates. But as all dynamic DNS information departed at the same time and is being
recreated as systems check into DHCP, it points to the issue being on the DNS side.

Dave

On 6/9/15 13:21, Cuttler, Brian (HEALTH) wrote:

> I disabled update_optimization and as dhcp releases are renewed (I already has update_static
> enabled) we are re-creating the DNS entries.
>
> I still have no idea why those entries all got deleted and am rather stressed over it.
>
> Please help me to understand what went wrong, and how to prevent or properly fix.
>
> Many thanks,
>
> Brian
>
> *From:* [hidden email] [mailto:[hidden email]] *On Behalf Of
> *Cuttler, Brian (HEALTH)
> *Sent:* Tuesday, June 09, 2015 12:05 PM
> *To:* Users of ISC DHCP
> *Subject:* dns_update_log shows dynamic entries deleted
>
> We are running on Centos 7.1 and have dhcp 4.2.5, with bind 9.9.4.
>
> We are running dhcp with subnet 10.57.36.0/22 permitted dynamic dns udpates, except for the last
> 100 addresses in the pool, which we excluded to allow for static addresses that we needed.
>
> Things have been working wonderfully – but at 08:18 this morning a large number of entries were
> removed from dynamic dns.
>
> The dhcp clients were either, only asleep for the night (Windows, printers) or actually online
> (linux).
>
> I’ve no idea why these DNS entries were removed, exactly how to get them back, and how to
> prevent them from being removed again.
>
> I’d suspect it was a lease expiration issue, but these machines all check in and renew, at least
> they should be, I can check the boot.log and see.
>
> Thanks for your help,
>
> Brian
>
>
>
> _______________________________________________
> dhcp-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/dhcp-users
>

--
Dave Calafrancesco
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: dns_update_log shows dynamic entries deleted

Doug Barton
On 6/9/15 12:05 PM, dave c wrote:
> Dynamic DNS updates are written to a local cache for the zone. I don't
> know if that cache will always persist through having DNS restarted, but
> I'd suggest that as it's dynamic, it's not designed to be persistent in
> the DNS zone. Otherwise it would be in a DNS zone file

Um, sorry, that's not how it works at all. :)

The name server doesn't care how the records are added to the zone, they
become part of the zone, and are written out to disk specifically so
that they survive a server restart.

Doug

--
I am conducting an experiment in the efficacy of PGP/MIME signatures.
This message should be signed. If it is not, or the signature does not
validate, please let me know how you received this message (direct, or
to a list) and the mail software you use. Thanks!


_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: dns_update_log shows dynamic entries deleted

Cuttler, Brian (HEALTH)
In reply to this post by dave c
Dave,

I had wondered that myself, but I haven't found any evidence to support the theory (and I would much rather have a theory than an unknown).

The bind server seemed to check point its leases hourly (seems to slip back just a little each hour), and while I do see a bind restart in the logs, it was one I triggered at 09:40, there is nothing to indicate server (host or named) at the time of the event (08:18). Nor any indication that DHCPD restarted prior to my manually restarting it later on.

Nor have a found a switch to remove (auto-purge) 'old' dynamic entries, in the named.conf, though I could have overlooked it, certainly I did nothing to enable such a switch.

It's a mystery to me.

Thanks,
Brian

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of dave c
Sent: Tuesday, June 09, 2015 3:06 PM
To: Users of ISC DHCP
Subject: Re: dns_update_log shows dynamic entries deleted

Well, you are likely asking the wrong side of the equation.

If I had to venture a guess, your DNS server reloaded itself.

Dynamic DNS updates are written to a local cache for the zone. I don't know if that cache will always persist through having DNS restarted, but I'd suggest that as it's dynamic, it's not designed to be persistent in the DNS zone. Otherwise it would be in a DNS zone file :)

Do you have any info or stats on the DNS side of things?

If you had said that you started seeing updates not getting through to DNS, then I'd suggest that the issue might be that the DHCP system and DNS system had a change in what they used to authenticate updates. But as all dynamic DNS information departed at the same time and is being recreated as systems check into DHCP, it points to the issue being on the DNS side.

Dave

On 6/9/15 13:21, Cuttler, Brian (HEALTH) wrote:

> I disabled update_optimization and as dhcp releases are renewed (I
> already has update_static
> enabled) we are re-creating the DNS entries.
>
> I still have no idea why those entries all got deleted and am rather stressed over it.
>
> Please help me to understand what went wrong, and how to prevent or properly fix.
>
> Many thanks,
>
> Brian
>
> *From:* [hidden email]
> [mailto:[hidden email]] *On Behalf Of *Cuttler,
> Brian (HEALTH)
> *Sent:* Tuesday, June 09, 2015 12:05 PM
> *To:* Users of ISC DHCP
> *Subject:* dns_update_log shows dynamic entries deleted
>
> We are running on Centos 7.1 and have dhcp 4.2.5, with bind 9.9.4.
>
> We are running dhcp with subnet 10.57.36.0/22 permitted dynamic dns
> udpates, except for the last
> 100 addresses in the pool, which we excluded to allow for static addresses that we needed.
>
> Things have been working wonderfully - but at 08:18 this morning a
> large number of entries were removed from dynamic dns.
>
> The dhcp clients were either, only asleep for the night (Windows,
> printers) or actually online (linux).
>
> I've no idea why these DNS entries were removed, exactly how to get
> them back, and how to prevent them from being removed again.
>
> I'd suspect it was a lease expiration issue, but these machines all
> check in and renew, at least they should be, I can check the boot.log and see.
>
> Thanks for your help,
>
> Brian
>
>
>
> _______________________________________________
> dhcp-users mailing list
> [hidden email]
> https://lists.isc.org/mailman/listinfo/dhcp-users
>

--
Dave Calafrancesco
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: dns_update_log shows dynamic entries deleted

Doug Barton
On 6/9/15 12:29 PM, Cuttler, Brian (HEALTH) wrote:
> Dave,
>
> I had wondered that myself, but I haven't found any evidence to support the theory (and I would much rather have a theory than an unknown).

Dave is wrong. :)

> The bind server seemed to check point its leases hourly (seems to slip back just a little each hour), and while I do see a bind restart in the logs, it was one I triggered at 09:40, there is nothing to indicate server (host or named) at the time of the event (08:18). Nor any indication that DHCPD restarted prior to my manually restarting it later on.

If the problem was caused by the DHCP server you would see the removals
in the logs. The BIND server did not spontaneously remove anything.

> Nor have a found a switch to remove (auto-purge) 'old' dynamic entries, in the named.conf, though I could have overlooked it, certainly I did nothing to enable such a switch.

There isn't one. :)

Doug

--
I am conducting an experiment in the efficacy of PGP/MIME signatures.
This message should be signed. If it is not, or the signature does not
validate, please let me know how you received this message (direct, or
to a list) and the mail software you use. Thanks!


_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: dns_update_log shows dynamic entries deleted

Simon Hobson
In reply to this post by dave c
dave c <[hidden email]> wrote:

> Dynamic DNS updates are written to a local cache for the zone. I don't know if that cache will always persist through having DNS restarted, but I'd suggest that as it's dynamic, it's not designed to be persistent in the DNS zone. Otherwise it would be in a DNS zone file :)

All wrong, sorry.
When you have a dynamic zone in BIND, then BIND does in fact update *the* zone file - including incrementing the SOA serial number. It does keep a journal file (<zone>.jnl), but that's for a different purpose.
This means that you cannot "edit the file and reload it" to manually update a dynamic zone - you'll lose your edits if you try. You have to do it one of 3 ways :
1) Use nsupdate
2) stop BIND, delete the journal file, edit the file, start BIND
3) Freeze the zone (rndc freeze domain.tld), edit the file, unfreeze the zone.

The journal file is used to keep multiple versions. When a slave requires a zone transfer to bring itself up to date, it will try and do an incremental transfer to save bandwidth. The master uses the information in the journal file to be able to send the increments from the zone the slave holds, and the current version held by the master.

Other than a small window (I suspect BIND does in fact cache some updates in memory for a short time before writing them to disk, and then there'll be OS caching), you can "pull the plug" on BIND and it won't lose updates.

_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users
Reply | Threaded
Open this post in threaded view
|

Re: dns_update_log shows dynamic entries deleted

Tony Finch
Simon Hobson <[hidden email]> wrote:
>
> Other than a small window (I suspect BIND does in fact cache some
> updates in memory for a short time before writing them to disk, and then
> there'll be OS caching), you can "pull the plug" on BIND and it won't
> lose updates.

No, DNS updates are durable. RFC 2136 section 3.5:

   When a zone is modified by an UPDATE operation, the server must
   commit the change to nonvolatile storage before sending a response to
   the requestor or answering any queries or transfers for the modified
   zone.

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
South Utsire: Northwesterly 4 or 5, occasionally 6 in southeast. Slight or
moderate. Rain later. Mainly good.
_______________________________________________
dhcp-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/dhcp-users