Last night my server backup process ran into the continua server project I was working on. It looks like that build process generated a SQL Server deadlock and then the whole web application has gone into melt down mode. I'm attempting to re-start the services and find out what's happened (server has also triggered a crash), so I'm not sure the exact source of this issue.
I've had to extract this directly from the SQL Server database.
Exception: GenericADOException
Message: could not execute query
[ SELECT TOP (@p0) Id5_, SourceEn2_5_, SourceEn3_5_, SourceEn4_5_, Date5_, Message5_, Hash5_, Importance5_, messaget9_5_, RepeatC10_5_, Resolved5_ FROM (select event0_.Id as Id5_, event0_.SourceEntityId as SourceEn2_5_, event0_.SourceEntityTypeId as SourceEn3_5_, event0_.SourceEntityName as SourceEn4_5_, event0_.Date as Date5_, event0_.Message as Message5_, event0_.Hash as Hash5_, event0_.Importance as Importance5_, event0_.messagetype as messaget9_5_, event0_.RepeatCount as RepeatC10_5_, event0_.Resolved as Resolved5_, ROW_NUMBER() OVER(ORDER BY event0_.Resolved, event0_.Date DESC) as __hibernate_sort_row from core_event event0_) as query WHERE query.__hibernate_sort_row > @p1 ORDER BY query.__hibernate_sort_row ]
Name:p1 - Value:0 Name:p2 - Value:20
[SQL: SELECT TOP (@p0) Id5_, SourceEn2_5_, SourceEn3_5_, SourceEn4_5_, Date5_, Message5_, Hash5_, Importance5_, messaget9_5_, RepeatC10_5_, Resolved5_ FROM (select event0_.Id as Id5_, event0_.SourceEntityId as SourceEn2_5_, event0_.SourceEntityTypeId as SourceEn3_5_, event0_.SourceEntityName as SourceEn4_5_, event0_.Date as Date5_, event0_.Message as Message5_, event0_.Hash as Hash5_, event0_.Importance as Importance5_, event0_.messagetype as messaget9_5_, event0_.RepeatCount as RepeatC10_5_, event0_.Resolved as Resolved5_, ROW_NUMBER() OVER(ORDER BY event0_.Resolved, event0_.Date DESC) as __hibernate_sort_row from core_event event0_) as query WHERE query.__hibernate_sort_row > @p1 ORDER BY query.__hibernate_sort_row]
Stack Trace: at NHibernate.Loader.Loader.DoList(ISessionImplementor session, QueryParameters queryParameters)
at NHibernate.Loader.Loader.ListIgnoreQueryCache(ISessionImplementor session, QueryParameters queryParameters)
at NHibernate.Hql.Ast.ANTLR.Loader.QueryLoader.List(ISessionImplementor session, QueryParameters queryParameters)
at NHibernate.Hql.Ast.ANTLR.QueryTranslatorImpl.List(ISessionImplementor session, QueryParameters queryParameters)
at NHibernate.Engine.Query.HQLQueryPlan.PerformList(QueryParameters queryParameters, ISessionImplementor session, IList results)
at NHibernate.Impl.SessionImpl.List(IQueryExpression queryExpression, QueryParameters queryParameters, IList results)
at NHibernate.Impl.AbstractSessionImpl.List(IQueryExpression queryExpression, QueryParameters parameters)
at NHibernate.Impl.ExpressionQueryImpl.List()
at NHibernate.Linq.DefaultQueryProvider.ExecuteQuery(NhLinqExpression nhLinqExpression, IQuery query, NhLinqExpression nhQuery)
at NHibernate.Linq.DefaultQueryProvider.Execute(Expression expression)
at NHibernate.Linq.DefaultQueryProvider.Execute[TResult](Expression expression)
at Remotion.Linq.QueryableBase`1.GetEnumerator()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
at Continua.Logging.EventManager.GetEvents(Paging paging)
at Continua.Services.CoreService.GetEvents(Paging paging)
at SyncInvokeGetEvents(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
Exception: SqlException
Message: Transaction (Process ID 61) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Stack Trace: at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at System.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows)
at System.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more)
at System.Data.SqlClient.SqlDataReader.Read()
at NHibernate.Driver.NHybridDataReader.Read()
at NHibernate.Loader.Loader.DoQuery(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.DoQueryAndInitializeNonLazyCollections(ISessionImplementor session, QueryParameters queryParameters, Boolean returnProxies)
at NHibernate.Loader.Loader.DoList(ISessionImplementor session, QueryParameters queryParameters)
Web site result
Event log
Could not initialise database:
System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, Boolean withFailover)
at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)
at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)
at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions)
at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
at System.Data.SqlClient.SqlConnection.Open()
at FluentMigrator.Runner.Processors.SqlServer.SqlServerProcessor..ctor(IDbConnection connection, IMigrationGenerator generator, IAnnouncer announcer, IMigrationProcessorOptions options, IDbFactory factory)
at FluentMigrator.Runner.Processors.SqlServer.SqlServer2008ProcessorFactory.Create(String connectionString, IAnnouncer announcer, IMigrationProcessorOptions options)
at FluentMigrator.Runner.Initialization.TaskExecutor.InitializeProcessor(String assemblyLocation)
at FluentMigrator.Runner.Initialization.TaskExecutor.Initialize()
at FluentMigrator.Runner.Initialization.TaskExecutor.Execute()
at Continua.Migrations.Runner.Execute(String database, String connectionString, String fileName)
at Continua.Server.Service.Migrator.Begin()
ClientConnectionId:00000000-0000-0000-0000-000000000000
Exiting...