Rebalance docs, Roadmap, etc

I'm sure this was asked before but I can't seem to find it in the
archives.

Is there an expected release date for httpclient & core 4.1?

-k



This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law.  If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED.  Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase & Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
 If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.

This is a multi-part message in MIME format.
Is there such a thing? Where might I be able to see what is planned for
Cassandra.

 

 

Anthony Ikeda

Java Analyst/Programmer

Cardlink Services Limited

Level 4, 3 Rider Boulevard

Rhodes NSW 2138

 

Web: www.cardlink.com.au | Tel: + 61 2 9646 9221 | Fax: + 61 2 9646 9283

 

 


**********************************************************************
This e-mail message and any attachments are intended only for the use of
the addressee(s) named above and may contain information that is privileged
and confidential. If you are not the intended recipient, any display,
dissemination, distribution, or copying is strictly prohibited.   If you
believe you have received this e-mail message in error, please immediately
notify the sender by replying to this e-mail message or by telephone to
(02) 9646 9222. Please delete the email and any attachments and do not
retain the email or any attachments in any form.
**********************************************************************

Hi

It's most likely starting to get to that point where we should start
to ponder about Camel 3.0.

I have created a wiki page to gather ideas
https://cwiki.apache.org/confluence/d...mel 3.0 - Roadmap

Currently I have only listed major goals such as shifting to JDK 1.6+
and Spring 3.0+ as min requirement.

I personally think we should try to keep the roadmap in a reasonable
size of work.
I doubt we have the effort to do a 1+ year man full time work on a
Camel 3.0 (which we did for 2.0)

Well now we got the wiki page as a starting point. So raise your voice.





Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

  I've been doing a clean up pass on the Jiras we opened for Geronimo 
3.0 work that are included on the roadmap summary page at:

https://cwiki.apache.org/GMOxDEV/java-ee-6-roadmap.html

Right now, I'm trying to get a better picture of work items that might 
still need to be implemented for the 3.0 release.  I've done a first 
pass and closed the Jiras that I'm fairly certain are complete.  If 
people could take a few minutes to look at the remaining issues 
(particularly ones assigned to you) and update those as well, this will 
make it easier to assess and prioritize the work that needs to be 
completed.

Looking at the Jiras, the area I have the biggest uncertainty with is 
the state of the bean validation work.  I believe a number of the bean 
validation items still need to be done, but it's not clear which ones 
are complete and which ones still need implementing.  Those would be 
Jiras 5230, 5221, 5222, 5223, 5226, and 5227.

In other areas, I think we're in pretty good shape, but comments in a 
number of Jiras hint at additional work that needs to be done.  At this 
point, I think it would be good to resolve the main issue and open new 
Jiras for cleanup items that still need to done.

Rick

Rebalance has been immensely useful to us. When adding new storage
nodes, we are able to mitigate the damage from HD failures, are able
to add nodes more sparingly, and just generally balances out load
across storage nodes.

( 12 storage nodes, 6 drives each if it helps )

On Wed, Jul 14, 2010 at 3:12 AM, dormando <dorm### @rydia.net>
wrote:
 Hey,

 Are there any of you out there who active use the existing
"rebalance"
 feature and have measureable benefits from it? Please confirm that
you
 aren't just running it because it felt like a good idea, and that you
 actually get results from it?

 I have a larger plan for rewriting rebalance by wiring it over the
new
 drain code, but I can also just get the new drain code out very
quickly
 which will fix many problems for many people.

 However in the process I might disable/destroy the existing rebalance
 code, and it'll stay that way until we can finish writing the
rebalance
 stuff on top of it.

 If there're enough complainers I'll try to not break the old code, or
just
 wait until I can replace all of it at once...

 Thanks,
 -Dormando

Hi,

I have a problem with enable_rebalance. It seems that after a while it
stopped
working. Now i read in Store.pm about List::Util::shuffle() not being
really
random, is this still true?

I've also setup a test environment to analyse the rebalance problems and
after
three days i'm still not very sure why it also seems to get stuck with only
14
files. The current device usage situation is as follows:

Checking devices...
  host device         size(G)    used(G)    free(G)   use%   ob state  
I/O%
  ---- ------------ ---------- ---------- ---------- ------ ----------
-----
  [ 1] dev1             0.176      0.055      0.121  31.01%  writeable  
0.0
  [ 1] dev2             0.176      0.074      0.102  42.16%  writeable  
0.0
  [ 2] dev4             0.176      0.128      0.048  72.51%  writeable  
0.0
  [ 3] dev3             0.176      0.123      0.053  70.03%  writeable  
0.0
  ---- ------------ ---------- ---------- ---------- ------
             total:     0.703      0.379      0.324  53.93%

where the devices are spread along the three hosts like this:

virtualmedia1 [1]: alive
                   used(G) free(G) total(G)
  dev1: alive      0.054   0.122   0.176  
  dev2: alive      0.073   0.103   0.176  

virtualmedia2 [2]: alive
                   used(G) free(G) total(G)
  dev4: alive      0.127   0.049   0.176  

virtualmedia3 [3]: alive
                   used(G) free(G) total(G)
  dev3: alive      0.123   0.053   0.176  


I've set enable_rebalance=1 and never gets reset. For all rebalance attemps
it says:

Rebalance for DevFID[d=3;f=36]
(http://192.168.210.1:7500/dev3/0/000/000/0000000036.fid) failed: no
suitable destination devices available

and it seems it tries to itterate over all fids endlessly.

Could someone possibly shed some light on this? In our production
environment
we have a new server on which each device is full for only 9% while the
other
two are filled around 80% and rebalance doesn't work there either. We
really
need this because we pay bandwith per server (if we consume too much) so we
really need to get this balanced.


thanks in advance,

Martijn

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I'm running mogilefs 2.30 with 14 hosts with 4 devices each. When I
!watch on one of my trackers, I see lots (several to many a minute) of
messages like these:

:: [replicate(9979)] Unable to create source socket to 10.2.128.90:7500
for /dev90128/0/670/784/0670784277.fid
:: [replicate(9979)] Failed copying fid 670784277 from devid 90128 to
devid 96208 (error type: src_error)
:: [replicate(9979)] copy_error: error copying fid 670784277 from devid
90128 during replication
:: [replicate(9977)] Unable to create source socket to 10.2.131.210:7500
for /dev210321/0/670/783/0670783216.fid
:: [replicate(9977)] Failed copying fid 670783216 from devid 210321 to
devid 96208 (error type: src_error)

I ran a fsck a while ago(now long completed, according to status) and
occasionally see lines like this:

:: [fsck(9970)] node 10.2.128.90 seems to be down in get_file_size
:: [fsck(9970)] Connectivity problem reaching device 90228 on host
10.2.128.90

very rarely do I actually see real monitor timeouts.

mogadm check shows the cluster is fairly bored with not much IO on the
hosts. The DB machine isn't overloaded, either. I'm using the zonelocal
and network plugins.

I'm also noticing that I have a significant number of files way over
replicated.  My replication policy has a max of 4 for any class,
however, out of my 212Mn files in mogilefs, about 2.5Mn have 10 or more
copies.  Many millions more are replicated 6 times or more.

I added some new nodes and ran a rebalanace.  After only a couple of
percents it would stop.  So, I start it again but it stops after a
couple more percents, repeat.

Any thoughts appreciated.

here's some more info:

!stats
uptime 8367694
pending_queries 0
processing_queries 0
bored_queryworkers 10
queries 2648773
work_queue_for_delete 70
work_queue_for_fsck 150
work_queue_for_replicate 10

!jobs
delete count 1
delete desired 1
delete pids 21633
fsck count 1
fsck desired 1
fsck pids 9970
job_master count 1
job_master desired 1
job_master pids 9971
monitor count 1
monitor desired 1
monitor pids 9986
queryworker count 10
queryworker desired 10
queryworker pids 533 1976 2957 12534 22680 24926 27103 27710 28132 29976
reaper count 1
reaper desired 1
reaper pids 22503
replicate count 5
replicate desired 5
replicate pids 9965 10016 14167 14317 31029


mogilefsd.conf:
db_dsn = DBI:mysql:blah:blah
local_network = 10.0.128.0/22

db_user = ...
db_pass = ...
listen = 0.0.0.0:7001
conf_port = 7001
listener_jobs = 10
delete_jobs = 1
replicate_jobs = 5
mog_root = /var/lib/mogdata
reaper_jobs = 1
plugins = ZoneLocal

mogstored.conf:
httplisten=0.0.0.0:7500
mgmtlisten=0.0.0.0:7501
docroot=/var/lib/mogdata

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFMbYNa+Idx1gGGQ1YRAgyEAJ9Rxjbo9ajioA3cb8iRJWJLpG19egCfXzxA
ot3kTHy2+5k5ZRmxpvWD1tw=
=e/jJ
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 8/23/10 12:24 PM, dormando wrote:
> It looks like the maxconns default is 10K and I don't set it
explicitly.
>  I assume that is connections that would show up in netstat?  I'm
seeing
> less than 1K active connections there.
>
> I'm even seeing this occasionally:
>
> ro### @a0100:/etc/mogilefs# mogadm check
> Checking trackers...
>   127.0.0.1:7001 ... REQUEST FAILURE (is the tracker up?)
> Unable to retrieve host information from tracker(s).
>
 
 Is it hovering around 1k as in suspiciously close to 1k? or well
below 1k?
 There's a chance that it would have failed to increase the maxconns
if not
 started from root or from a user with adjusted maxconns.

Actually, I just changed my methodology slightly to weed out the
TIME_WAITS and other stuff and now run:

netstat -anp | grep mogstored | wc -l

and now see only 300-400 sockets open and still see timeouts when I !watch

 
 when you run mogadm check, is it failing immediately or does it feel
like
 a timeout?

When it fails, which is probably only 5% of the time, it seems to do so
pretty quickly, which could be in about 2s.  When I normally do a
"mogadm check" there's a bit of a pause before it returns "ok" and when
it fails the pause length is about the same amount of time.







-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFMcwP6+Idx1gGGQ1YRAmnjAJ4qL+0yWP/gPcxP+rJPI4rlRCWIbACfQuLr
TfXVy4kxju3kvaatxo/cXrU=
=ohD5
-----END PGP SIGNATURE-----

Author: jfclere
Date: Thu Jul 22 10:32:38 2010
New Revision: 966567

URL: http://svn.apache.org/viewvc?rev=966567&view=rev
Log:
Release 6.0.29

Modified:
    tomcat/site/trunk/docs/download-60.html
    tomcat/site/trunk/docs/index.html
    tomcat/site/trunk/docs/oldnews.html
    tomcat/site/trunk/xdocs/download-60.xml
    tomcat/site/trunk/xdocs/index.xml
    tomcat/site/trunk/xdocs/oldnews.xml

Modified: tomcat/site/trunk/docs/download-60.html
URL:
http://svn.apache.org/viewvc/tomcat/s...567&view=diff

Author: jfclere
Date: Thu May 27 11:36:12 2010
New Revision: 948789

URL: http://svn.apache.org/viewvc?rev=948789&view=rev
Log:
Add something about the Tomcat track at ApacheCon.

Added:
    tomcat/site/trunk/docs/images/ACNA2010_120x240_basic.gif   (with
props)
    tomcat/site/trunk/xdocs/images/ACNA2010_120x240_basic.gif   (with
props)
Modified:
    tomcat/site/trunk/docs/index.html
    tomcat/site/trunk/xdocs/index.xml

Added: tomcat/site/trunk/docs/images/ACNA2010_120x240_basic.gif
URL:
http://svn.apache.org/viewvc/tomcat/s...789&view=auto

Author: poirier
Date: Thu Jun  3 15:13:07 2010
New Revision: 951027

URL: http://svn.apache.org/viewvc?rev=951027&view=rev
Log:
Rebuild site after updating subversion links

Modified:
    httpd/site/trunk/docs/apreq/docs/libapreq2/apreq_install.html
    httpd/site/trunk/docs/dev/devnotes.html
    httpd/site/trunk/docs/mod_fcgid/index.en.html
    httpd/site/trunk/docs/mod_ftp/index.en.html
    httpd/site/trunk/docs/mod_mbox/index.html
    httpd/site/trunk/docs/mod_smtpd/index.html
    httpd/site/trunk/docs/test/index.html

Modified: httpd/site/trunk/docs/apreq/docs/libapreq2/apreq_install.html
URL:
http://svn.apache.org/viewvc/httpd/si...027&view=diff

Author: markt
Date: Fri Jul  9 12:14:49 2010
New Revision: 962498

URL: http://svn.apache.org/viewvc?rev=962498&view=rev
Log:
CVE-2010-2227

Modified:
    tomcat/site/trunk/docs/security-5.html
    tomcat/site/trunk/docs/security-6.html
    tomcat/site/trunk/docs/security-7.html
    tomcat/site/trunk/xdocs/security-5.xml
    tomcat/site/trunk/xdocs/security-6.xml
    tomcat/site/trunk/xdocs/security-7.xml

Modified: tomcat/site/trunk/docs/security-5.html
URL:
http://svn.apache.org/viewvc/tomcat/s...498&view=diff

Author: rjung
Date: Sun Jul 25 21:08:15 2010
New Revision: 979120

URL: http://svn.apache.org/viewvc?rev=979120&view=rev
Log:
Adding sub second timestamps and request end time to mod_log_config.

Add special format tokens to %{...}t. The extended syntax allows the
form: "WHICH:WHAT".

WHICH is either:
- "begin": use the time when the request started
- "end": take "now" as the time
You can omit WHICH, default is "begin".
If you omit WHICH, the separating column is not allowed.

WHAT is either:
- "sec": timestamp in Unix seconds
- "msec": timestamp in Unix milliseconds
- "msec_frac": millisecond fraction of the Unix timestamp,
               3 digits, 0-padded
- "usec": timestamp in Unix microseconds
- "usec_frac": microsecond fraction of the Unix timestamp
               6 digits, 0-padded
- anything different from those tokens: use strftime()
You can omit WHAT, default is the formatted timestamp as
used by the Common Log Format.

The implementation uses a new request_config for mod_log_config
to pass the request end time around between different calls to
log formatters, but the end time is only generated if needed.

Modified:
    httpd/httpd/trunk/docs/manual/logs.html.en
    httpd/httpd/trunk/docs/manual/logs.xml
    httpd/httpd/trunk/docs/manual/mod/mod_log_config.html.en
    httpd/httpd/trunk/docs/manual/mod/mod_log_config.xml
    httpd/httpd/trunk/modules/loggers/mod_log_config.c

Modified: httpd/httpd/trunk/docs/manual/logs.html.en
URL:
http://svn.apache.org/viewvc/httpd/ht...120&view=diff

Author: kkolinko
Date: Fri Jun 18 00:31:03 2010
New Revision: 955799

URL: http://svn.apache.org/viewvc?rev=955799&view=rev
Log:
Correct links

Modified:
    tomcat/site/trunk/docs/bugreport.html
    tomcat/site/trunk/docs/getinvolved.html
    tomcat/site/trunk/xdocs/bugreport.xml
    tomcat/site/trunk/xdocs/getinvolved.xml

Modified: tomcat/site/trunk/docs/bugreport.html
URL:
http://svn.apache.org/viewvc/tomcat/s...799&view=diff

Author: rpluem
Date: Mon Jul 12 06:57:41 2010
New Revision: 963195

URL: http://svn.apache.org/viewvc?rev=963195&view=rev
Log:
Merge r955966 from trunk:

* Ensure that we only return a non fatal error if the request is
idempotent
  and if we did not sent any request body so far.

Submitted by: rpluem
Reviewed by: rpluem, trawick, niq

Modified:
    httpd/httpd/branches/2.2.x/   (props changed)
    httpd/httpd/branches/2.2.x/STATUS
    httpd/httpd/branches/2.2.x/docs/   (props changed)
    httpd/httpd/branches/2.2.x/docs/conf/mime.types   (props changed)
    httpd/httpd/branches/2.2.x/modules/proxy/mod_proxy_ajp.c
    httpd/httpd/branches/2.2.x/support/ab.c   (props changed)
    httpd/httpd/branches/2.2.x/support/suexec.c   (props changed)

Propchange: httpd/httpd/branches/2.2.x/

Author: trawick
Date: Mon May 24 14:26:49 2010
New Revision: 947651

URL: http://svn.apache.org/viewvc?rev=947651&view=rev
Log:
odd assortment of spelling fixes

Modified:
    httpd/httpd/trunk/STATUS
    httpd/httpd/trunk/docs/manual/rewrite/flags.xml
    httpd/httpd/trunk/include/util_ldap.h
    httpd/httpd/trunk/modules/lua/docs/basic-configuration.txt
    httpd/httpd/trunk/modules/metadata/mod_setenvif.c
    httpd/httpd/trunk/modules/proxy/mod_proxy_fcgi.c
    httpd/httpd/trunk/server/mpm/event/event.c
    httpd/httpd/trunk/server/mpm/winnt/mpm_winnt.c

Modified: httpd/httpd/trunk/STATUS
URL:
http://svn.apache.org/viewvc/httpd/ht...651&view=diff

Author: jorton
Date: Mon Aug  2 13:03:04 2010
New Revision: 981498

URL: http://svn.apache.org/viewvc?rev=981498&view=rev
Log:
- add description of CVE-2010-2791

Modified:
    httpd/site/trunk/docs/security/vulnerabilities-oval.xml
    httpd/site/trunk/docs/security/vulnerabilities_22.html
    httpd/site/trunk/xdocs/security/vulnerabilities-httpd.xml

Modified: httpd/site/trunk/docs/security/vulnerabilities-oval.xml
URL:
http://svn.apache.org/viewvc/httpd/si...498&view=diff

Author: markt
Date: Fri Jul  9 09:41:25 2010
New Revision: 962466

URL: http://svn.apache.org/viewvc?rev=962466&view=rev
Log:
Documentation improvements:
- Explicit version number in <title ../>
- Add version and date to every page
- Remove duplicate "Apache Tomcat 7" and promote real page title
- Automate copyright end date

Modified:
    tomcat/trunk/build.xml
    tomcat/trunk/webapps/docs/architecture/project.xml
    tomcat/trunk/webapps/docs/config/project.xml
    tomcat/trunk/webapps/docs/project.xml
    tomcat/trunk/webapps/docs/tomcat-docs.xsl

Modified: tomcat/trunk/build.xml
URL:
http://svn.apache.org/viewvc/tomcat/t...466&view=diff

Author: mjc
Date: Mon Jul 26 08:56:12 2010
New Revision: 979202

URL: http://svn.apache.org/viewvc?rev=979202&view=rev
Log:
2.2.16 announcement went out so do a quick explanation of the flaw

Modified:
    httpd/site/trunk/docs/security/vulnerabilities-oval.xml
    httpd/site/trunk/docs/security/vulnerabilities_22.html
    httpd/site/trunk/xdocs/security/vulnerabilities-httpd.xml

Modified: httpd/site/trunk/docs/security/vulnerabilities-oval.xml
URL:
http://svn.apache.org/viewvc/httpd/si...202&view=diff

Author: covener
Date: Sat May 29 20:19:10 2010
New Revision: 949436

URL: http://svn.apache.org/viewvc?rev=949436&view=rev
Log:
mod_authnz_ldap: Allow the initial DN lookup to bind with a
transformation of the basic auth username.


Modified:
    httpd/httpd/trunk/CHANGES
    httpd/httpd/trunk/docs/manual/mod/mod_authnz_ldap.html.en
    httpd/httpd/trunk/docs/manual/mod/mod_authnz_ldap.xml
    httpd/httpd/trunk/modules/aaa/mod_authnz_ldap.c

Modified: httpd/httpd/trunk/CHANGES
URL:
http://svn.apache.org/viewvc/httpd/ht...436&view=diff