Saturday 10 May 2014

Development Team Size

DISCLAIMER: I am writing about team, that starts a project. Not about support team or a team which has to add pieces of functionality to existing application.

Narrative

Recently we started two new small[had to be] applications in the bank, that are small transformer/bridge, and aggregator(integrity controller by function).

We started with aggregator - two of us. We did a proof of concept and working app with most of functionality fairly quickly. Pairing most of the time, quickly discussing problems. If we had different vision - we could quickly find a consensus (usually with well-known phrase "let's quickly spike and then change if we don't like";)

Than the priorities has changed (surprise-surprise), and we started transformer. With one small difference: now there was a big team (9 expierenced developers, including us). All but myself were working in one team on the other project before, so people knew and respected each other.

But something in our cooperation was broken from the very beginning. We usually make team decisions about any new library, technology, the way to organize process and structure the codebase. So we have a discussion about any topic like one of those and at the end we are voting/deciding on the way.

Sounds great in theory. Turned into disaster in practice.

So many senior developers on a small (at the beginning) codebase - means everyone has his own ideas how to do things. And this is a start of new application - so everyone saw that as an ideal opportunity to apply his ideas and practices from the empty page.
Was hard to agree at the beginning on anything.

Some team members were very upset when their point of view didn't get enough supporters in voting.

Some people where rewriting bits using their approach, and then others were refactoring the same place back because they saw that as an issue.

We came back to the same discussions again and again. Too many opinionated people, not enough codebase size to make everyone work on his own feature.

Theory: small green-field application - what could be better for developers?
Result: slow delivery; team not happy.

Conclusion


Someone would argue: the situation turned to worse because there was no single authority/architect/team-lead, who would drive decisions.
Maybe. But this will not make people happy, it will make them to obey but they will keep their creativity unrealised.

I think the ideal solution here would be to start application with 2 developers - smallest possible team. And once the skeleton and main features would be ready - add more people for bug-fixing and to plug in new features.
This approach will give first 2 devs ability to build the prototype and to evolve design quickly, without spending too much time for holy-wars and discussions.
This will reduce project budget and the freedom to throw away the code if initial assumptions were wrong or business has changed it's mind.
It is much easier to find 2 developers, that will have more or less similar ideas and approach. It is much harder to find 5-7 with the same mindset.
This 2 devs can pair on idea, than split to quickly implement different parts, then pair again on next piece of functionality, and so on.
They can (and should) seat next to each other, in order to be able to quickly gather near one monitor for a discussion and to split again.

Why not 1? Illness, personal matters, vacations and so on of the single developer should not stop the project - 2 can always go for a rest one after another.

Why not 3? If there is any problem that has few options for solution - usually voting in this case will decide the way. Than usually it would be 2 against 1 and that one will be not happy. In case of team of 2 - they know there is no way to vote, so they have to learn to find consesus using facts and arguments. Once they will learn each other - they will make decisions very quickly. Its always easier in case of 2, because there is no way to split into groups with different opinion, and there is no way to escape.

4 and more... Well, the same issues as for case of 3.

I really hope the managers will find out at one point that more people doesn't always mean faster delivery.
And too many smart people on the same task can result in problem.

Sunday 20 May 2012

Atomikos and Oracle AQ: connections pool problem



Don't use Atomikos jms ConnectionFactoryBean pooling with Oracle AQ as they recommend in documentation! You will not use the pool indeed. How to use it right?

In example from Atomikos documentation we are creating oracleDataSource and AQJmsFactory uses it to create XA factory.
And Atomikos JMS factory bean works like an adapter/decorator for this XA factory to make it transaction-aware and to create connection pool:


oracle.jdbc.xa.client.OracleXADataSource  xaDataSource = new oracle.jdbc.xa.client.OracleXADataSource();
XAQueueConnectionFactory xaQueueConnectionFactory = AQjmsFactory.getXAQueueConnectionFactory ( xads );
AtomikosConnectionFactoryBean pooledQueueConnectionFactoryBean = new AtomikosConnectionFactoryBean();
pooledQueueConnectionFactoryBean.setXaConnectionFactory ( xaQueueConnectionFactory ); 
//...set pool settings in pooledQueueConnectionFactoryBean ...


But behind the scene if you will profile application which uses this code, you will find that Atomikos have no access to JMS connection to use it in pool(naturally, as Oracle AQ uses jdbc connection instead) and each closeSession delegates to Oracle AQ conn factory which calls PhysicalConnection.close() on jdbc connection. And pool is not working as expected. If your app has few listeners and each of them calls receive frequently you will create/close jdbc connections each time and performance would be horrible.

The problem also is that you can't enable connection caching in OracleXADataSource and use it's pool(if you will do this AQjmsFactory will throw exception), and dbcp and c3p0 pools are not working either as AQ doesn't work with proxy this implementations create around instance of oracle connection.

The only solution which works for us is to use atomikos xaDataSourceBean and than non-xa jms queue connection factory:

oracle.jdbc.xa.client.OracleXADataSource xaDataSource = new oracle.jdbc.xa.client.OracleXADataSource();
AtomikosDataSourceBean pooledDataSourceBean = new AtomikosDataSourceBean();
pooledDataSourceBean.setXaDataSource(xaDataSource);
//...set pool settings in pooledDataSourceBean ...
QueueConnectionFactory queueConnectionFactory = AQjmsFactory.getQueueConnectionFactory ( pooledDataSourceBean );


This will allow you to use XA-aware data source and atomikos connection pool now will work, as it is between AQ JMS connection factory and db data source.

Monday 9 April 2012

OOP Patterns in functional language

The topic was inspired by my Master Thesis, which I have handed in recently but I want to proceed working on it as haven't done it properly.

The thing I tried to do - to implement OOP patterns in functional language - Erlang.
Resulted sources are at github

A lot of developers will admit that it doesn't worth the attempt because of difference in paradigms and because most of OOP patterns are already implemented naturally in FP languages.

I was trying to describe my goals in thesis(paper)  and below is some part from introduction:

Today it is realy hard to find software developer who hadn't heard about design patterns and refactoring techniques.
This terms became very popular few years ago and are very widely used in modern software development.
I remember when I heard the first time at second year of study about patterns and read few articles about it - I was very delighted with this new world and interesting methods of code organization and become very enthusiastic about patterns.
The problem which really caused me to choose the subject for thesis is that "Design Patterns" in our mind is very closely associated with Object-Oriented paradigm.
Since the time of GoF book design patterns where very popular subject for different researches, discussions and pulications, they had a lot of heads involved and thats why the topic was developed very quickly and have now a huge base.
But the problem was that most of this researches and publications were about implementation and application of patterns/refactoring to Object-Oriented paradigm. It was most popular and authors used it as primary in their work. Other paradigms had no such careful attention, and it is black side of OOP monopolism in enterprise application development.
I always have been a loyal supporter of functional paradigm, so I aways wanted to have this gap filled at least in this paradigm.
In order to do classic patterns in functional languages we should escape from OO paradigm limitations. As well we should have new patterns suggested by functional languages.
After developing for over four years in at work(mostly Java), I was pretty comfortable with best practices for creating applications in OO languages.
But when I tried to create my first application in Erlang(am not taking into account Lisp samples written during very short academic course), I ran into all sorts of problems. You can feel it even trying to write any complex XSLT with parsing and recursion.
But I have realized that what I was trying to do was use an OO approach in a functional programming language. There were fundamental differences between the two approaches, and "paradigm shift" is not just yet another philosophic phrase.
Another idea is that to make functional paradigm more popular and widely used we need to involve to it a part of experienced OOP "Coryphaeus". Because functional programming encompasses a very different way of composing programs, programmers who are used to the imperative paradigm can find it difficult to learn.
So I think that it would be very useful to show comparison/make mapping between well-known OOP design patterns and functional analogues, and to create last ones if we need that.
In process of making such comparison we would find out problems and gaps of each paradigm in regard to patterns and refactoring techniques.
Hope it would be helpful for students as well when they try to learn course in functional languages. As at such courses we usually do not have time to look at FP in depth, and are only learning basic syntax. This fact causes us not to think about FP as something which can be used in real life, in production.
But as far as most of students are aware of OOP design principles and patterns, it would be interesting for them to find out that the same problems/tasks can be solved in functional languages in much easier and straight-forward way.