Thursday, September 13, 2018

SQL CDC fail after servicepack upgrade



Issue
·         SQL CDC job fail
·         Database log space usage can’t clear

Troubleshooting
·         SQL CDC job fail with error message
o   Msg 8144, Level 16, State 2, Procedure or function sp_batchinsert_1582628681 has too many arguments specified. For more information, query the sys.dm_cdc_errors dynamic management view
o   Msg 8144, Level 16, State 2, Procedure or function sp_batchinsert_690101499 has too many arguments specified. For more information, query the sys.dm_cdc_errors dynamic management view
·         Error message in sqlerror log
o   Could not open database [xxxxxx]. Replication settings and system objects could not be upgraded. If the database is used for replication, run sp_vupgrade_replication in the [master] database when the database is available.
·         Error in sys.dm_cdc_errors
session_id
phase_number
entry_time
error_number
error_severity
error_state
error_message
1
7
9/11/18 2:20
8144
16
2
Procedure or function sp_batchinsert_1389248004 has too many arguments specified.
1
7
9/11/18 2:20
22863
16
1
Failed to insert rows into Change Data Capture change tables. Refer to previous errors in the current session to identify the cause and correct any associated problems.
2
7
9/11/18 2:22
8144
16
2
Procedure or function sp_batchinsert_1389248004 has too many arguments specified.
2
7
9/11/18 2:22
22863
16
1
Failed to insert rows into Change Data Capture change tables. Refer to previous errors in the current session to identify the cause and correct any associated problems.
3
7
9/11/18 2:24
8144
16
2
Procedure or function sp_batchinsert_1389248004 has too many arguments specified.
3
7
9/11/18 2:24
22863
16
1
Failed to insert rows into Change Data Capture change tables. Refer to previous errors in the current session to identify the cause and correct any associated problems.
4
7
9/11/18 2:26
8144
16
2
Procedure or function sp_batchinsert_1389248004 has too many arguments specified.
4
7
9/11/18 2:26
22863
16
1
Failed to insert rows into Change Data Capture change tables. Refer to previous errors in the current session to identify the cause and correct any associated problems.
5
7
9/11/18 2:28
8144
16
2
Procedure or function sp_batchinsert_1389248004 has too many arguments specified.
5
7
9/11/18 2:28
22863
16
1
Failed to insert rows into Change Data Capture change tables. Refer to previous errors in the current session to identify the cause and correct any associated problems.
6
7
9/11/18 2:30
8144
16
2
Procedure or function sp_batchinsert_1389248004 has too many arguments specified.
6
7
9/11/18 2:30
22863
16
1
Failed to insert rows into Change Data Capture change tables. Refer to previous errors in the current session to identify the cause and correct any associated problems.

·         log_reuse_wait_desc for the database = REPLICATION
·         Log space usage for xxxxxx = 80% of 320 gb
·         Run Log backup for xxxxxx and log usage can’t be clear

Finding
·         If a CDC enabled database is part of an Always On availability group and users follow the general recommendations of upgrading the secondary replica first, sp_vupgrade_replication will not run in such databases during upgrade because secondary replica databases are not in read/write mode. This is a known behavior and is by design.

Workaround
·         Run sp_vupgrade_replication in master db and issue solve
Reference

Thursday, July 05, 2018

Cluster log


Sample Error :

1. cluster-node-keeps-loosing-communication-to-file-share-disk-witness
Failed to arbitrate for \\FileServer\ShareFolder\16d280e9-633c-4db3-a7f5-2db3b4fae331 with 64.

net helpmsg 64
The specified network name is no longer available.

Extract from fourm :
This is a typical heartbeat communication failed between some nodes. Mostly the network connection or communication is failed among the cluster nodes. 
Event 1135 will be logged on all nodes in the Cluster except for the node that was removed. The reason for this event is because one of the nodes in the Cluster marked that node as down. It then notifies all of the other nodes of the event. When the nodes are notified, they discontinue and tear down their heartbeat connections to the downed node.

 having-a-problem-with-nodes-being-removed-from-active-failover-cluster-membership