Best unofficial Apache Server developers community
Username
Forgot password?
Sign in with Twitter account
Sign in with Facebook account
List archives

ngx_http_read_client_request_body() causing random timeouts

Content does not get replaced
(70 lines)
Fastcgi: core dump was caused by duplicated request header
(23 lines)
May 31, 2011
Srebrenko Šehić
Srebrenko Šehić
Hi,

I've been trying to identify what is causing timeouts towards the
client when a POST request is sent to nginx. I have a custom module
that reads the POST payload and runs in ACCESS phase. It's based on
nginx-form-input module from agentz.

Normally, the modules works OK, but it sometimes times out and nginx
logs the following:

2011/05/31 09:31:16 [alert] 25731#0: *1 zero size buf in output t:1
r:0 f:0 0000000204DFD000 0000000204DFD000-0000000204DFD000
0000000000000000 0-0 while sending request to upstream, client:
192.168.0.27, server: 192.168.0.27, request: "POST /slurp_md5.php
HTTP/1.1", upstream: "http://192.168.0.9:80/slurp_md5.php", host:
"192.168.0.27"

I've noticed that this only happens with keep-alive connections.

Can anybody shed some light on this? I've looked at other modules and
nginx core, but can't seem to figure what could be causing this.

Thanks. Code for the sample module is enclosed below. nginx is vanilla
1.0.3 (with perl module compiled).

Cheers,
Srebrenko

---- module source code ----

#include <ngx_config.h>
#include <ngx_core.h>
#include <ngx_http.h>

static ngx_int_t ngx_http_dummy_post_read_init(ngx_conf_t *cf);
static ngx_int_t ngx_http_dummy_post_read_access_handler(ngx_http_request_t
*r);

typedef struct {
    ngx_flag_t    done:1;
    ngx_flag_t    waiting_more_body:1;
} ngx_http_dummy_post_read_ctx_t;

static ngx_http_module_t  ngx_http_dummy_post_read_module_ctx = {
    NULL,                                  /* preconfiguration */
    ngx_http_dummy_post_read_init,         /* postconfiguration */
    NULL,                                  /* create main configuration */
    NULL,                                  /* init main configuration */
    NULL,                                  /* create server configuration
*/
    NULL,                                  /* merge server configuration */
    NULL,                                  /* create location configuration
*/
    NULL                                   /* merge location configuration
*/
};


ngx_module_t  ngx_http_dummy_post_read_module = {
    NGX_MODULE_V1,
    &ngx_http_dummy_post_read_module_ctx,            /* module context
*/
    NULL,                                  /* module directives */
    NGX_HTTP_MODULE,                       /* module type */
    NULL,                                  /* init master */
    NULL,                                  /* init module */
    NULL,                                  /* init process */
    NULL,                                  /* init thread */
    NULL,                                  /* exit thread */
    NULL,                                  /* exit process */
    NULL,                                  /* exit master */
    NGX_MODULE_V1_PADDING
};

void
ngx_http_dummy_post_read_payload_handler(ngx_http_request_t *r)
{
    ngx_http_dummy_post_read_ctx_t     *ctx;

    r->read_event_handler = ngx_http_request_empty_handler;

    ctx = ngx_http_get_module_ctx(r, ngx_http_dummy_post_read_module);

    ctx->done = 1;

    r->main->count--;

    if (ctx->waiting_more_body) {
        ctx->waiting_more_body = 0;

        ngx_http_core_run_phases(r);
    }
}

static ngx_int_t
ngx_http_dummy_post_read_access_handler(ngx_http_request_t *r)
{
    ngx_int_t                          rc;
    ngx_http_dummy_post_read_ctx_t *ctx;

    ctx = ngx_http_get_module_ctx(r, ngx_http_dummy_post_read_module);

    if (ctx != NULL) {

        if (ctx->done) {
            return NGX_DECLINED;
        }

        return NGX_DONE;
    }

    if (r->method != NGX_HTTP_POST) {
        return NGX_DECLINED;
    }

    ctx = ngx_palloc(r->pool, sizeof(ngx_http_dummy_post_read_ctx_t));

    if (ctx == NULL) {
        ngx_http_finalize_request(r, NGX_ERROR);
        return NGX_ERROR;
    }

    ngx_http_set_ctx(r, ctx, ngx_http_dummy_post_read_module);

    rc = ngx_http_read_client_request_body(r,
ngx_http_dummy_post_read_payload_handler);

    if (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE) {
        return rc;
    }

    if (rc == NGX_AGAIN) {
        ctx->waiting_more_body = 1;

        return NGX_DONE;
    }

    return NGX_DECLINED;
}

static ngx_int_t
ngx_http_dummy_post_read_init(ngx_conf_t *cf)
{
    ngx_http_handler_pt        *h;
    ngx_http_core_main_conf_t  *cmcf;

    cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module);

    h =
ngx_array_push(&cmcf->phases[NGX_HTTP_ACCESS_PHASE].handlers);
    if (h == NULL) {
        return NGX_ERROR;
    }

    *h = ngx_http_dummy_post_read_access_handler;

    return NGX_OK;
}


Reply
Tags: input modulepayload
Messages in this thread
ngx_http_read_client_request_body() causing random timeouts
Resolved: (HDFS-1286) Dry entropy pool on Hudson boxes causing test timeouts
December 20, 2010 11:24:49 PM
[ https://issues.apache.org/jira/browse/HDFS-1286?page=com.atlassian.jir a.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Boudnik resolved HDFS-1286.
During unittest, changes not applied causing random test failure.
January 25, 2011 03:43:00 AM
In my python unit test, I have tests that randomly failed. They look like this: 1. db = pymongo.Connection() 2. assert db.profile.find_one({'_id': 1})['c'] == 2 3. db.profile.update({'_id': 1}, {'$inc': {'c': 1}}) 4. assert…
client connection timeouts vs. thrift timeouts
March 29, 2011 02:17:33 PM
I've been scratching my head on this one for a day now and I'm hoping someone can help clear it up. The initial question was: does it make sense to have a configurable connection timeout (for a client connecting to a cassandra server) separate…
Timeouts
February 10, 2011 08:16:29 AM
Hi, in preparation for an infrastructure change that will allow us to shard and use replica sets, I setup an arbiter and config server on our master mongo server yesterday. I made a mistake bringing up the arbiter and forgot to pass an option to…
ActiveMQ Timeouts
March 21, 2011 11:43:31 AM
Hello, I currently have an ActiveMQ broker that we use with both Perl (Stomp) and C++ (Openwire). I am trying to find a solution for timeouts. Here is what I have tried. Perl - Stomp consumers receive heartbeat messages from a seperate producer.…
connection timeouts
September 8, 2010 07:59:33 AM
I am using apache directory studio 1.5.2 There is a very annoing issue with connections. If I leave connection open for a long time open it gots frozen and it takes a long time to revive it. Maybe it is possible to add some feature to connection…
connection timeouts
September 8, 2010 07:41:21 AM
I am using apache directory studio 1.5.2 There is a very annoing issue with connections. If I leave connection open for a long time open it gots frozen and it takes a long time to revive it. Maybe it is possible to add some feature to connection…
Exchange Timeouts
October 20, 2010 12:32:58 PM
How do I configure (total) timeouts for exchanges? I have different components in the routes, and I do not want to configure component timeouts. What I need is, to configure a total timeout so that any exchange would stop/exit the route if it…
question about RS to DN timeouts
April 4, 2011 04:43:58 PM
<property> <name>hbase.client.pause</name> <value>1000</value> <description>General client pause value. Used mostly as value to wait before running a retry of a failed get, region lookup,…
timeouts while rebalancing
December 20, 2010 01:51:13 PM
Hi, I just added a 4th node in my 0.10.1 + innostore cluster and I am seeing all kind of timeouts both for retrieving objects and trying to execute riak-admin status which gives me: RPC to 'ria### @x.x.x' failed: {'EXIT', …
C# slaveOk timeouts
February 6, 2011 08:55:45 PM
Hi, We're using 1.6.5 server with 2 active nodes in a replset using the C# driver v0.11. Everything works fine until we add "slaveok=true" to the connection string. Once added the client times out when trying to connect to the server. All the…
timeouts on transactions etc?
May 28, 2011 10:26:16 PM
Does Postgres have any mechanisms where one can set an activity timeout, say either dynamically thru SQL to affect a current session, or alternately in a configuration file so to take effect globally? I mean for example so we can tell Postgres…
Protocol Buffer Timeouts
July 30, 2010 11:48:53 AM
Hello all, We're doing some load testing using Riak erlang protocol buffers interface and are running into timeouts. From the looks of it, the timeouts are coming from the gen_server:calls in riakc_pb_socket.erl which don't specify a timeout,…
read timeouts in JAX-RS function
May 11, 2011 01:53:41 PM
CXF 2.4.0. I have a JAX-RS method that accepts a POST of type application/json. I've written the function to take an InputStream as an argument. Every so often, it fails with a read time-out. The client passes the entire contents of the posted…
Timeouts Running Deletes
January 24, 2011 11:26:11 AM
Hi, we're using Mongo to manage some web session data, and I'm maintaining a backlog of 7 days. To keep the dataset from spiraling out of control, I run nightly deletes via a cron script. Unfortunately, we're getting a lot of timeout errors when…
Turn off all timeouts with WebClient in JAX-RS
January 31, 2011 10:00:39 AM
How can I use the WebClient API and crank the timeout to infinity?
recipient list and timeouts
April 26, 2011 12:23:55 PM
I am a new developer working with Camel. We need to route messages using a parallel processed recipient list. We are using an aggregation strategy to combine the results into one message. Our requirements include the need to know which specific…
Problems with timeouts, restated
January 14, 2011 06:23:53 AM
Last night I realized that the various messages I've sent about my timeout woes reflect some conclusions I've drawn that may, in fact, not be correct. So I write to start over. The original setup was as follows (2.3.1): Server side is very plain…
Fwd: read timeouts in cassandra 0.6.5
November 5, 2010 10:20:00 AM
Hi, I have a simple keyspace: <Keyspace Name="reef-test"> <ColumnFamily Name ="Meas" CompareWith="LongType" /> <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawa reStrategy</ReplicaPlacementStrategy> …
Cassandra timeouts under low load
June 15, 2010 11:00:04 AM
Hi, I'm running cassandra .6.2 on a dedicated 4 node cluster and I also have a dedicated 4 node hadoop cluster. I'm trying to run a simple map reduce job against a single column family and it only takes 32 map tasks before I get floods of thrift…
MySQL: How to retrieve a random row or multiple random rows?
January 3, 2011
I have a MySQL database table that stores the URLs of photos. I need to pull 5 random records from the database of a particular type. I can pull 5…
Intermittent DNS timeouts
May 6, 2011
We have a LAN with two firewalls, two adsl lines. The firewall (192.168.4.2) which routes to Provider A is giving us problems. It is used for…
Intermittent DNS timeouts, independent of DNS provider
February 17, 2011
This is a problem that has plagued my office for a few months now. It has bounced between our ISP (Comcast), our IT contractor, and even our VOIP…
Dns timeouts - resolv.conf wrong?
February 22, 2011
hello, i have a question concerning the correct setup of debian's resolv.conf file. i have 3 domains: a-domain.com, b-domain.com and c-domain.com…
MongoDB Cursor Timeouts while doing a lot of writes
May 9, 2011
We have a cluster of 2 replica Sets, with 3 servers per set. With a single collection being sharded. We also have a quite a few more(8+) collections…
FTP server timeouts on passive, I can only use active.
April 8, 2011
I am running Proftpd on Ubuntu Server 10.04. Filezilla was configured to switch to active mode on failure but connection fails at MLSD time out;…
Avoiding DNS timeouts when a dns server fails
January 4, 2011
Hi there. We have a small datacenter with about a hundred hosts pointing to 3 internal dns servers (bind 9). Our problem comes when one of the…
Avoiding locked SQLite database timeouts with PHP/PDO
May 24, 2011
I'm currently trying to fix an issue in Drupal core's issue queue having to do with timeouts happening when someone visits a site while it is…
I keep getting Varnish Guru Mediation 503 errors despite setting timeouts
February 4, 2011
I have the following backend setup: backend seanesopenko { .host = "seanesopenko.ca"; .port = "8080"; .connect_timeout = 60s; .first_byte_timeout =…
What's the proper way to handle timeouts for active record with a connection pool?
February 6, 2011
I've tracked down a strange error undefined method `run_callbacks' for nil:NilClass and been able to reproduce it with this sample code. Basically…
How should I handle server timeouts and error code responses to an http post in Android App?
May 9, 2011
Hi Folks, My Android App does http posts to URLs like http://example.com/abc.php?email=abc@xyz.com So the Android App basically talks to PHPs on the…
RSA is causing me headaches
March 1, 2011
Situation I want to encrypt/decrypt byte[] data using RSA in my C# porject. I use OpenSSL.Crypto this is my code byte[] msg =…
Dashes In the Url Causing 404 Error
January 25, 2011
I'm working in PHP (5.2.14) on an Apache Server (1.3.33) with MySQL, and a problem has emerged with the pretty url's that produced by our .htaccess…
What's causing this forced redirect?
May 20, 2011
I do not have root access to the site site.com but I have access to a subdomain me.site.com I add an index.php (or any file and or folder for that…
Mod_rewrite causing too many redirects
May 7, 2011
The .htaccess we have in our document_root directory is as follows: RewriteEngine on RewriteRule ^posts/1/first-post-title$ /posts/1.html [L]…
What is causing this error when I try to install the pg gem?
March 3, 2011
I'm trying to install the pg gem . bens-macbook-pro:liferecord ben$ bundle config build.pg --with-pg-config=/users/ben/postgresql/bin…
Saving to a mysql db from c# causing an exception
March 15, 2011
I am using a WPF window and trying to save an item into the database. I am working with three mysql database tables: Orders , Items , and Meals .…
Htaccess code causing 500 error
May 13, 2011
Hi, I'm following a book tutorial (Effortless Ecommerce by Larry Ullman) to build an ecommerce site. When I add the following code to my htaccess…
.htaccess causing redirect loop
December 28, 2010
I have a URL on my site that works without a / but when I add a / it says there's a redirect loop... I'm struggling to see the problem here...…
Redirecting Pages with PHP causing problems
February 21, 2011
I have a page which has a link to a php page which takes data from $_GET and updates a database. After that it returns the user to the homepage…