A visual python debugger

pudb, it looks impressive from the picture below, though I seldom use any python debugger.

New pub/sub system-Apache Kafka

Apache Kafka, it is from linkedin with a lot of specific characters. It aims at providing a unified stream for both real-time and offline consumption. It stores data in the file sequentially without random access. The status of consumer is stored in client side and the consumer works in pull way. It uses zero-copy method like sendfile (FileChannel.transferTo/transferFrom) to reduce copy cost. It really looks a good design. I think it's worth having a try. Here is a video about this. http://sna-projects.com/blog/2011/08/kafka/

Functional Thinking

A great video about functional programming Functional Thinking

The main idea is “Think about results, not steps”. It also covers more details about FP.

How to model this pattern?

I am trying to improve the intra-day executions. For example, I wanna sell a stock but its mid price keeps monotone decreasing/increasing/unchanging at most of time (the small graph). occasionally, it has abrupt price fluctuation (the big graph). imbalance=ask depth - bid depth of top 5 price levels. Do you have any suggestion to model this to get good executions assuming time frame is 10 minutes? Are MACD hist and EMA applicable here? How to combine imbalance and price indicator?

Substitute for sprintf

snprintf/sprintf are widely used in our code. In one program, it takes up >30% cpu usage from kcachegrind report. It is necessary to rewrite sprintf for performance concern. In stead of sprintf, I prefer C++ stream style. Such a simple implementation as follows is almost 3X faster.

typedef std::pair<const char*, size_t> StrN;
typedef std::pair<size_t, size_t> NN; 
template <size_t L>
class StrStream
{
public:
  StrStream() : p_(buf_) {}
  size_t length() const { return p_ - buf_; }
  const char* get() const { return buf_; }
  void reverse(char* a, char* b)
  {
    while (a < b) std::swap(*(a++), *(b--));
  }
  template <typename T> StrStream& operator<<(T n)
  {
    assert(n>=0);
    char* p = p_; 
    do {
      *(p_++) = n % 10 + '0';
    } while (n /= 10);
    reverse(p, p_-1);
    return *this;
  }
  StrStream& operator<<(char c)  
  {
    *(p_++) = c;
    return *this;
  }
  StrStream& operator<<(StrN p)
  {
    while (p.second-- > 0) { // no *str check
      *(p_++) = *(p.first++);
    }
    return *this;
  }
  StrStream& operator<<(const char* str)
  {
    while (*str) {
      *(p_++) = *(str++);
    }
    return *this;
  }
  StrStream& operator<<(char* str) { return operator<<((const char*)str); }
  StrStream& operator<<(NN n)
  {
    assert(n.first>=0);
    char* p = p_;
    do {
      *(p_++) = n.first % 10 + '0';
    } while (n.first /= 10);
    char* p2 = p + n.second;
    while (p_ < p2) *(p_++) = '0';
    reverse(p, p_-1);
    return *this;
  }

private:
  char buf_[L];
  char* p_;
};

int main() {
  clock_t t = clock();
  for (size_t i = 0; i < 10000000; ++i) {
    char buf[256];
    sprintf(buf, "%ld %04ld %04ld %.4s", i, i, i, "abcde");
  }
  std::cout << (clock() -t) << std::endl;
  t = clock();
  for (size_t i = 0; i < 10000000; ++i) {
    StrStream<256> s;
    s << i << ' ' << NN(i, 4) << ' ' << NN(i, 4) << ' ' << StrN("abcde", 4);
  }
  std::cout << (clock() -t) << std::endl;
  return 0;
}

13830000
3560000