Using Regex in Embedded Pig in Java

All,

 

I am using pig embedded in Java and need to use matches in my pig job.
However when I try to use escape characters in the pig line, the
compiler complains. How do I use complex regex while embedding?

 

Sample code that is throwing errors:

 

myServer.registerQuery("filtered = FILTER firstcut BY dIP matches
'\Q34.21.12.*\E';");

 

error: invalid escape sequence.

 

Thanks,

 

Matt

 

Hi, everyone
    The following rule comes from
rules/base_rules/modsecurity_crs_41_sql_injection_attacks.conf , but I
don't understand what does the regular expression "(?:[\\\(\)\%#]|--)"
mean. What's the meaning of "\%" in a regex?

SecRule MATCHED_VAR "(?:[\\\(\)\%#]|--)"
        
"t:none,setvar:'tx.msg=%{rule.msg}',setvar:tx.sql_injection_score=+%{tx.critical_anomaly_score},setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},setvar:tx.%{rule.id}-WEB_ATTACK/SQL_INJECTION-%{matched_var_name}=%{tx.0}"

Well, how do I use the content of a variable in regex?

$username = "user1"
file {  "userdata.tar.bz2":
                source => "puppet://$server/modules/$module/
userdata.tar.bz2",
                ensure => $users ? {
                                /$username/ => absent,
                                default => present,
                        },
}

$users is a custom fact that contains all local users:

users => at avahi bin daemon dnsmasq ftp games haldaemon lp mail
messagebus nobody ntp polkituser postfix pulse root sshd suse uuidd
wwwrun man news uucp puppet user1

When I hardcode "user1" into the regex my test works fine and the file
is removed.

But things like /$variable/ or /\$variable/ or /#{variable}/ just
don't work.
Is it even possible in version 0.25.4?

I am trying to match groups of nodes - i.e.

Node: synd1-path2.path2.some.domain
Node: synd2-path2.path2.some.domain

By using either of the node definitions below:

node /^synd\w+\.path2\.some\.domain$/ {
    include ibapps
    include db
}

Hi  Jesse and All,
I have written some simple benchmarks for harmony regex and find the
performance of harmony is poor compared to RI. For example, Mathcer.find()
only reach 60% of that of RI. I heard Android use icu4jni re-implement
this
module. Since icu4jni use native code I think it may has higher
performance
than harmony. I am trying to use icu4jni as the back-end of harmony regex
but find icu4jni has no functions related to regex operations.
I know there are some android guys in our community. So can anyone tell me
some detail info for android's regex, like if it re-implement the regex
logic using native code by android itself rather than icu4jni and really
get
higher performance compared to harmony regex? Thanks a lot!

---------- Forwarded message 

Hi all,

I was wondering if it's possible to validate a <html:password> field
on client-side by using a regular expression.
A look at validateMask.js told me that the only field types supported are
'hidden', 'text', 'textarea' and 'file'.

Code:

            if ((field.type == 'hidden' ||
                field.type == 'text' ||
                 field.type == 'textarea' ||
                                 field.type == 'file') &&
                 (field.value.length > 0)) {

                if (!jcv_matchPattern(field.value, oMasked[x][2]("mask")))
{
                    if (i == 0) {
                        focusField = field;
                    }
                    fields[i++] = oMasked[x][1];
                    isValid = false;
                }
            }


Is this the exspected behavior? Is there any reason for not supporting
'password' fields?
If so, how would I validate a password field by using a regex (without
changing the js file on my own :))?

Cheers,


Ref. 324 * Geoinformationszentrum
Tel. 0211 9449-6310 * Fax: 0211 9449-6610
Email: stephan.### @it.nrw.de<mailto:stephan.k### @it.nrw.de>

Update AWS S3 log format deserializer regex

Kevin,

During the merge to java6 I did a --record-only merge (which just
updates the metadata to say that it is complete but changes nothing)
for this commit on the assumption that the changes were already made in
r958904.  If that is not correct - i.e. there is work missing - then
please
make a commit with the missing changes.

Thanks,
 Mark.


In message <20100629101401### @eris.apache.org>,
zhou### @apache.org 
writes:

 Author: zhoukevin
 Date: Tue Jun 29 10:14:01 2010
 New Revision: 958912
 
 URL: http://svn.apache.org/viewvc?rev=958912&view=rev
 Log:
 As to java.util.Properties.store(os, comments) method, if the
comments argume
 nt is not null, then '#' character and a line separator are first
written to 
 the output stream. Thus, the comments can serve as an identifying
comment. An
 y '\n', '\r' or "\r\n" in comments is replaced by a line separator
generated 
 by the Writer and if the next character in comments is not character
# or cha
 racter ! then an ASCII # is written out after that line separator.
 This patch fixes the above issue. In addition, it includes several
test cases
  for coverage.
 
 Modified:
    
harmony/enhanced/java/trunk/classlib/modules/luni/src/main/java/java/util
 /Properties.java
    
harmony/enhanced/java/trunk/classlib/modules/luni/src/test/api/common/org
 /apache/harmony/luni/tests/java/util/PropertiesTest.java
 
 Modified:
harmony/enhanced/java/trunk/classlib/modules/luni/src/main/java/jav
 a/util/Properties.java
 URL:
http://svn.apache.org/viewvc/harmony/...k/classlib/module

s/luni/src/main/java/java/util/Properties.java?rev=958912&r1=958911&r2=958912
 &view=diff
 

Regis,

This breaks the build for the IBM VME (from developerWorks).  Since they
don't have a sun.misc.Unsafe, so the AtomicInteger can't be resolved.

Any ideas how to fix this?

Also, the luni.jar manifest says:

  java.util.concurrent;resolution:=optional,

I wonder when it becomes non-optional.  Personally, I'd say breaking
java.io.File would be enough to make it mandatory.

Regards,
 Mark.

In message <20100705030517### @eris.apache.org>,
regi### @apache.org
writes:

 Author: regisxu
 Date: Mon Jul  5 03:05:16 2010
 New Revision: 960424
 
 URL: http://svn.apache.org/viewvc?rev=960424&view=rev
 Log:
 make File.createTempFile thread-safe to avoid to return the same file
multipl
 e times
 
 File.counter could be accessed by multiple threads, so use
AtomicInteger to m
 ake
 sure each thread using different int value to create temp file.
 
 Modified:
    
harmony/enhanced/java/trunk/classlib/modules/luni/src/main/java/java/io/F
 ile.java
 
 Modified:
harmony/enhanced/java/trunk/classlib/modules/luni/src/main/java/jav
 a/io/File.java
 URL:
http://svn.apache.org/viewvc/harmony/...k/classlib/module

s/luni/src/main/java/java/io/File.java?rev=960424&r1=960423&r2=960424&view=di
 ff
 

Author: markt
Date: Thu Jul 22 22:48:18 2010
New Revision: 966888

URL: http://svn.apache.org/viewvc?rev=966888&view=rev
Log:
Tab police: javax.servlet (partial)
Thanks to Checkstyle and Eclipse source formatting

Modified:
    tomcat/trunk/java/javax/servlet/Filter.java
    tomcat/trunk/java/javax/servlet/FilterChain.java
    tomcat/trunk/java/javax/servlet/FilterConfig.java
    tomcat/trunk/java/javax/servlet/GenericServlet.java
    tomcat/trunk/java/javax/servlet/RequestDispatcher.java
    tomcat/trunk/java/javax/servlet/Servlet.java

Modified: tomcat/trunk/java/javax/servlet/Filter.java
URL:
http://svn.apache.org/viewvc/tomcat/t...888&view=diff

I've not benched it but this looks like a reasonable amount of code to
run each time the hostname is checked against this permission.

Maybe not too bad for getting these addresses from the interface, but if
they are checked for each connect...

Is it worth caching the full form of the IPv6 address in the InetAddress
itself instance?

Regards,
Tim

On 09/Jun/2010 15:05, hind### @apache.org wrote:
 Author: hindessm
 Date: Wed Jun  9 14:05:56 2010
 New Revision: 953015
 
 URL: http://svn.apache.org/viewvc?rev=953015&view=rev
 Log:
 Applying patches from "[#HARMONY-6532]
[classlib][luni]SocketPermission
 does NOT support compressed IPV6 address, should pass in full form
 address".
 
 Modified:
    
harmony/enhanced/java/trunk/classlib/modules/luni/src/main/java/java/net/NetworkInterface.java
    
harmony/enhanced/java/trunk/classlib/modules/luni/src/test/api/common/org/apache/harmony/luni/tests/java/net/NetworkInterfaceTest.java
 
 Modified:
harmony/enhanced/java/trunk/classlib/modules/luni/src/main/java/java/net/NetworkInterface.java
 URL:
http://svn.apache.org/viewvc/harmony/...015&view=diff
 

Author: markt
Date: Sun Jul 11 16:29:25 2010
New Revision: 963103

URL: http://svn.apache.org/viewvc?rev=963103&view=rev
Log:
Remove unnecessary code since Class.getCanonicalName() is available

Modified:
    tomcat/trunk/java/org/apache/jasper/compiler/Generator.java
    tomcat/trunk/java/org/apache/jasper/compiler/JspUtil.java

Modified: tomcat/trunk/java/org/apache/jasper/compiler/Generator.java
URL:
http://svn.apache.org/viewvc/tomcat/t...103&view=diff

In message <20100721145605### @eris.apache.org>,
tell### @apache.org
writes:

 Author: tellison
 Date: Wed Jul 21 14:56:05 2010
 New Revision: 966265
 
 URL: http://svn.apache.org/viewvc?rev=966265&view=rev
 Log:
 Fix problems with PriorityQueue:
 1 - the remove test should use equality using equals() not the result
of the 
 comparator being 0;
 2 - attempting to remove an object of the 'wrong' type should just
return
 false, not throw a class cast exception.

Tim,

There is a non-trivial conflict in the merge of this commit from trunk
to the java6 branch - due to the fact that this appears to have been
fixed there already but with an equals method call the opposite way
around.

Would you mind doing the merge for this commit.  The command will be
something like:

  svn merge -c 966265 --accept postpone \
            https://svn.apache.org/repos/asf/harmony/enhanced/java/trunk

Regards,
 Mark.

In message <2010071410370### @eris.apache.org>,
ode### @apache.org
writes:

 Author: odeakin
 Date: Wed Jul 14 10:36:59 2010
 New Revision: 964001
 
 URL: http://svn.apache.org/viewvc?rev=964001&view=rev
 Log:
 To match the RI's behaviour, resolve and store the hostname when
getHostName 
 is called. Add a flag so we do not resolve the hostname for every
call to get
 HostName. Also add a regression test for the behaviour.

 [SNIP]

 ---
harmony/enhanced/java/trunk/classlib/modules/luni/src/test/api/common/org
 /apache/harmony/luni/tests/java/net/InetSocketAddressTest.java
(original)
 +++
harmony/enhanced/java/trunk/classlib/modules/luni/src/test/api/common/org
 /apache/harmony/luni/tests/java/net/InetSocketAddressTest.java Wed
Jul 14 10:
 36:59 2010
 @@ -31,7 +31,9 @@ public class InetSocketAddressTest exten
      public void test_ConstructorLjava_lang_StringI() throws
Exception {
          // regression test for Harmony-1042
          InetSocketAddress address = new
InetSocketAddress("127.0.0.1", 0);
 -        assertNotNull(address.getHostName());
 +        assertEquals("/127.0.0.1:0", address.toString());
 +        assertEquals("localhost", address.getHostName());
 +        assertEquals("localhost/127.0.0.1:0", address.toString());
      }
      
      /**

I would have assumed that the removed:

  assertNotNull(address.getHostName());

was not:

  assertEquals("localhost", address.getHostName());

for good reasons[0].

Therefore it might be better to make the new code:

    assertEquals("/127.0.0.1:0", address.toString());
    String localhostName = address.getHostName();
    assertNotNull(localhostName);
    assertEquals(localhostName+"/127.0.0.1:0", address.toString());

or something like that?

Regards,
 Mark.

[0] Such as "localhost" being a rather arbitrary string that may be
    different on different machines?  Some older Debian machines
    have "localhost.localdomain" first in their /etc/hosts entries IIRC.

Author: markt
Date: Wed Jul 14 22:15:50 2010
New Revision: 964222

URL: http://svn.apache.org/viewvc?rev=964222&view=rev
Log:
Remove the ExpiresActive option. Just comment it out if you don't want to
use it.

Modified:
    tomcat/trunk/java/org/apache/catalina/filters/ExpiresFilter.java
    tomcat/trunk/java/org/apache/catalina/filters/LocalStrings.properties
    tomcat/trunk/test/org/apache/catalina/filters/TestExpiresFilter.java
    tomcat/trunk/webapps/docs/config/filter.xml

Modified: tomcat/trunk/java/org/apache/catalina/filters/ExpiresFilter.java
URL:
http://svn.apache.org/viewvc/tomcat/t...222&view=diff

Author: markt
Date: Sat Jul 10 21:43:02 2010
New Revision: 962923

URL: http://svn.apache.org/viewvc?rev=962923&view=rev
Log:
Eclipse warnings for annotations

Modified:
    tomcat/trunk/java/javax/annotation/PostConstruct.java
    tomcat/trunk/java/javax/annotation/PreDestroy.java
    tomcat/trunk/java/javax/annotation/security/DenyAll.java
    tomcat/trunk/java/javax/annotation/security/PermitAll.java

Modified: tomcat/trunk/java/javax/annotation/PostConstruct.java
URL:
http://svn.apache.org/viewvc/tomcat/t...923&view=diff

Author: markt
Date: Fri Jun 18 11:19:38 2010
New Revision: 955937

URL: http://svn.apache.org/viewvc?rev=955937&view=rev
Log:
Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=49443
Use remoteIp... rather then remoteIP... consistently

Modified:
    tomcat/trunk/java/org/apache/catalina/filters/RemoteIpFilter.java
    tomcat/trunk/java/org/apache/catalina/valves/RemoteIpValve.java
    tomcat/trunk/webapps/docs/config/filter.xml
    tomcat/trunk/webapps/docs/config/valve.xml

Modified:
tomcat/trunk/java/org/apache/catalina/filters/RemoteIpFilter.java
URL:
http://svn.apache.org/viewvc/tomcat/t...937&view=diff

Author: markt
Date: Mon Jul  5 20:51:21 2010
New Revision: 960692

URL: http://svn.apache.org/viewvc?rev=960692&view=rev
Log:
Fix https://issues.apache.org/bugzilla/show_bug.cgi?id=49528
Previous fix was incomplete. Improve test case and fix
TCK and test cases pass with this patch

Modified:
    tomcat/trunk/java/org/apache/catalina/connector/Request.java
    tomcat/trunk/java/org/apache/catalina/core/AsyncContextImpl.java
    tomcat/trunk/test/org/apache/catalina/core/TestAsyncContextImpl.java

Modified: tomcat/trunk/java/org/apache/catalina/connector/Request.java
URL:
http://svn.apache.org/viewvc/tomcat/t...692&view=diff

Author: markt
Date: Thu Jul 22 22:35:32 2010
New Revision: 966883

URL: http://svn.apache.org/viewvc?rev=966883&view=rev
Log:
Tab police: javax.el
Thanks to Checkstyle

Modified:
    tomcat/trunk/java/javax/el/ArrayELResolver.java
    tomcat/trunk/java/javax/el/ListELResolver.java
    tomcat/trunk/java/javax/el/MapELResolver.java
    tomcat/trunk/java/javax/el/PropertyNotFoundException.java
    tomcat/trunk/java/javax/el/ResourceBundleELResolver.java

Modified: tomcat/trunk/java/javax/el/ArrayELResolver.java
URL:
http://svn.apache.org/viewvc/tomcat/t...883&view=diff