Connection handshake failed: Unexpected or badly formatted message.

Hello,

I am unable to send a message to a target service on a different SQL Server instance. Using the SQL Profiler, I get the following error from the target machine:

Broker::Connection: Connection handshake failed: An OS call failed: (80090326) 0x80090326(The message received was unexpected or badly formatted.). State 106.

On the initiator side, the transmission_status shows the following error:

An error occurred while receiving data: '64(The specified network name is no longer available.)'.

Any ideas on why the target thinks the message is "unexpected or badly formatted"? Btw, I am specifying the default message type and contract for the services.

Thanks.

The problem is in the NTLM/Kerberos authentification sequence and I reckon I never saw it before. If you only want to get this to work, you can change the endpoint authentication from WINDOWS to CERTIFICATE. If you want to see why is this failing in the first place, I suggest you follow up using the MS Connect site at https://connect.microsoft.com/SQLServer/Feedback?wa=wsignin1.0. A netmon capture of the handhsake sequence would be needed to investigate this.

Connection handshake failed.

Hi.

I created two instances of SQL Server Dev Edition on the same machine.

The two instances acted as the sender / receiver.

However, when the message is transmitted from "Sender" to "Receiver", the following errors are
displayed in "SQL Profiler".

「Connection handshake failed. There is no compatible authentication protocol. State 21.」

How should be dealt with with this?
My best regards.

piknik

Seems like there is a problem with how you've setup your endpoints. Could you paste the script which you used for creating the endpoints? If you don't have that, could you paste the results from this query run on both instances?

select * from sys.service_broker_endpoints

|||

One machine is configured to use CERTIFICATE authentication, the other is configured to use WINDOWS. (or one is configured to use WINDOWS KERBEROS and the other WINDOWS NTLM).

Configure the endpoints to have at least one compatible authentication protocol and it will work.

HTH,
~ Remus

|||

Thank you for the reply.

I used following Script.

[Sender]

USE master
GO

-- Create master key in the master database
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password62374'

-- Create the certificate for transport security
CREATE CERTIFICATE TransportCert1
FROM FILE = 'C:\Documents and Settings\ryutai\Desktop\Demo\Certs\TransportCert1.cer'
WITH PRIVATE KEY (
FILE = 'C:\Documents and Settings\ryutai\Desktop\Demo\Certs\TransportCert1.pvk',
DECRYPTION BY PASSWORD = 'password62374'
)
ACTIVE FOR BEGIN_DIALOG = ON
GO

-- Create a user in the master db to be associated with
-- the public key from the remote certificate (TestCert2.cer)
CREATE LOGIN remcert WITH PASSWORD = 'password62374'
CREATE USER remcert FOR LOGIN remcert

-- remcert has to have connect priviliges
GRANT CONNECT TO remcert

-- Install the public key from the remote cert in master
CREATE CERTIFICATE TransportCert2
AUTHORIZATION remcert
FROM FILE = 'C:\Documents and Settings\ryutai\Desktop\Demo\Certs\TransportCert2.cer'
ACTIVE FOR BEGIN_DIALOG = ON
GO

-- Switch to the user database
USE SuperMarketServer
GO

-- Create a master key
CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password62374'

-- Create a certificate for dialog security
CREATE CERTIFICATE DialogCert1
FROM FILE = 'C:\Documents and Settings\ryutai\Desktop\Demo\Certs\DialogCert1.cer'
WITH PRIVATE KEY (
FILE = 'C:\Documents and Settings\ryutai\Desktop\Demo\Certs\DialogCert1.pvk',
DECRYPTION BY PASSWORD = 'password62374'
)
ACTIVE FOR BEGIN_DIALOG = ON
GO

-- Create user that holds the remote public key for the dialog security certificate
CREATE USER remcert FOR LOGIN remcert

CREATE CERTIFICATE DialogCert2
AUTHORIZATION remcert
FROM FILE = 'C:\Documents and Settings\ryutai\Desktop\Demo\Certs\DialogCert2.cer'
ACTIVE FOR BEGIN_DIALOG = ON
GO

-- Create a remote service binding associating the remcert user
-- with the remote service
CREATE REMOTE SERVICE BINDING [SellItemBinding]
TO SERVICE 'SellItemService'
WITH USER = remcert,
ANONYMOUS = Off

-- The user has to have SEND permissions
GRANT SEND ON SERVICE::[SoldItemService] TO remcert
go

-- Enable communication between instances by creating an endpoint
CREATE ENDPOINT ServerEndpoint
STATE = STARTED
AS TCP
(
LISTENER_PORT = 5024
)
FOR SERVICE_BROKER (AUTHENTICATION = CERTIFICATE TransportCert1)

-- Finally grant connect permission to user used to secure the dialog
USE master
GO

GRANT CONNECT ON ENDPOINT::ServerEndpoint TO remcert
GO

SELECT * FROM sys.endpoints
--

It is set to "CERTIFICATE authentication" as which Sender and Reciver are the
same.

The sample of Script is put.
http://enterpriselibrary.jp/SampleDemo.zip

My Best Regards.

piknik.

Connection handshake failed - easiest possible configuration

Hi there!

Often discussed, but not really solved in my opinion - the connection between the partners and the witness causes problems.

My case: Three Servers in the same domain, three endpoints on 5022 with windows negotiation, all endpoints can be reached by telnet from each server. Mirrorring works. So far so good.

But one of these partners is not able to connect to the witness. The witness' error log is full with that:

"2006-06-01 13:45:20.32 Logon Database Mirroring login attempt failed with error: 'Connection handshake failed. An OS call failed: (8009030c) 0x8009030c(Der Anmeldeversuch ist fehlgeschlagen.). State 67.'. [CLIENT: 130.143.205.54]"

My Endpoints are created like

CREATE ENDPOINT [EASYRIS_Mirroring]

AUTHORIZATION [code1\dephbrsaa1-sys108]

STATE=STARTED

AS TCP (LISTENER_PORT = 5022, LISTENER_IP = ALL)

FOR DATA_MIRRORING (ROLE = PARTNER, AUTHENTICATION = WINDOWS NEGOTIATE

, ENCRYPTION = SUPPORTED ALGORITHM RC4);

What catches my eyes is that

GRANT CONNECT ON ENDPOINT::EASYRIS_Mirroring TO [code1\dephbrsaa1-sys108];

doesn't cause these user to appear in the result set of

SELECT EP.name, SP.STATE,

CONVERT(nvarchar(38), suser_name(SP.grantor_principal_id))

AS GRANTOR,

SP.TYPE AS PERMISSION,

CONVERT(nvarchar(46),suser_name(SP.grantee_principal_id))

AS GRANTEE

FROM sys.server_permissions SP , sys.endpoints EP

WHERE SP.major_id = EP.endpoint_id

ORDER BY Permission,grantor, grantee;

By the way, these mentioned user is sysadmin and grantor.

Has anyone an idea?

Torsten

So,

The 8009030c error from the OS indicates that there is a login error at the OS level. SQL isn't involved with the networking protocol yet. So, look at the credentials that SQL Server is running under. There may need to be a restart of the SQL Server process to pick up the new credentials.

Thanks,

Mark

|||

Hi Mark,

thanks, that was the missing information. I could resolve the issue:

It seems that 1. the SQL Server Processes of each partner and the wittness has to run under an equal domain user and 2. these domain user must be local admin.

Can someone confirm these thesis?

Thanks a lot, Torsten

|||

You do not have to run all the same accounts and run as the SA to setup mirroring. It is just that the easiest way to setup mirroring is to have all the accounts be the same and SA.

You can use different accounts on the servers, but they need to be granted access to the other endpoints.

You can also run as local system accounts, but you need to setup certificattes. It is all in BOL.

Thanks,

Mark