Hi,
in the last few weeks we have been having problems with our Continua installation. Often when queueing builds we receive database timeouts.
The full entry from Continua's event log looks like this:
Importance: Error
Date: Monday, August 11, 2014
Time: 12:10:18 PM
Message:
Unhandled Service Exception
Exception: GenericADOException
Message: could not insert: [Continua.Modules.Builds.BuildRepository#d00baa2f-516f-4fa2-85b7-a38400c87ed0][SQL: INSERT INTO builds_buildrepository (CacheRevision, Branch, Branchdisplayname, RepositoryId, BuildId, id) VALUES (?, ?, ?, ?, ?, ?)]
Stack Trace: at NHibernate.Persister.Entity.AbstractEntityPersister.Insert(Object id, Object[] fields, Boolean[] notNull, Int32 j, SqlCommandInfo sql, Object obj, ISessionImplementor session)
at NHibernate.Persister.Entity.AbstractEntityPersister.Insert(Object id, Object[] fields, Object obj, ISessionImplementor session)
at NHibernate.Action.EntityInsertAction.Execute()
at NHibernate.Engine.ActionQueue.Execute(IExecutable executable)
at NHibernate.Engine.ActionQueue.ExecuteActions(IList list)
at NHibernate.Engine.ActionQueue.ExecuteActions()
at NHibernate.Event.Default.AbstractFlushingEventListener.PerformExecutions(IEventSource session)
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
at NHibernate.Impl.SessionImpl.Flush()
at NHibernate.Transaction.AdoTransaction.Commit()
at Continua.Shared.Data.Hibernate.NHibernateDatabase.SaveChanges()
at Continua.Modules.Builds.BuildManager.QueueUsersBuild(Configuration configuration, User user, BuildPriority priority, IDictionary`2 variables, BuildChangesetAssociationOptions associationOptions, IDictionary`2 repositoryBranches, String comment)
at Continua.Modules.Builds.Services.ConfigurationService.Start(Guid configurationId, BuildPriority priority, BuildChangesetAssociationOptions associationOptions, String comment, IDictionary`2 variables, IDictionary`2 repoBranches)
at SyncInvokeStart(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
Exception: NpgsqlException
Message: A timeout has occured. If you were establishing a connection, increase Timeout value in ConnectionString. If you were executing a command, increase the CommandTimeout value in ConnectionString or in your NpgsqlCommand object.
Stack Trace: at Npgsql.NpgsqlState.ProcessBackendResponsesEnum(NpgsqlConnector context)
at Npgsql.NpgsqlCommand.GetReader(CommandBehavior cb)
at Npgsql.NpgsqlCommand.ExecuteNonQuery()
at NHibernate.AdoNet.AbstractBatcher.ExecuteNonQuery(IDbCommand cmd)
at NHibernate.AdoNet.NonBatchingBatcher.AddToBatch(IExpectation expectation)
at NHibernate.Persister.Entity.AbstractEntityPersister.Insert(Object id, Object[] fields, Boolean[] notNull, Int32 j, SqlCommandInfo sql, Object obj, ISessionImplementor session)
Is this a known problem? If so is there anything I can do to prevent these?
Whenever we receive one of these Errors only restarting the Continua services seems to help.
Regards,
Christoph
Hi Christoph, thank you for posting. We are sorry to hear that your having an issue here.
Timeouts can be caused by a number of factors (we don’t currently have any open issues in this area). In order to assist further could please get some additional information off you.
When you start receiving such errors would could you please send the following details to support@finalbuilder.com:
Log into your Continua server, connect to your PostgreSQL install (we have included pgadmin3 as part of the Continua install at ‘/VSoft Technologies/ContinuaCI/Server/PostgreSQL/pgAdmin3.exe’) and run:
SELECT * FROM pg_stat_activity
SELECT * FROM pg_stat_user_tables
SELECT COUNT(*) from core_privatemessage.
Whilst you are on the server could you please also let us know;
how much CPU / RAM/ Diskspace you have free (for example, can you see a run a way process, or something else taking up large CPU cycles)
how many concurrent builds are running when the issue occurred
how are the builds structures (how many repo’s etc)
the size of your C:\ProgramData\VSoft\Continua folder
The Version of Continua you are running, are you running the bundled PostgreSQL install ?
Do you have any scheduled backup or Virus scanning software that occurs around these times (can you see a pattern to the issue)
If the issue is happening regularly would you please be able to run a build with full debugging enabled (http://wiki.finalbuilder.com/display/continua/Debug+Logging).
If you are running the beta could we please have a diagnostics dump (http://SERVERNAME/administration/diagnosticsreport)
Once we have these details we can look closer to better understand the issue.
Thank you.